-- Logs begin at Wed 2022-09-14 00:52:09 UTC, end at Wed 2022-09-14 01:30:04 UTC. -- Sep 14 01:16:28.623339 dv-cinder-857075-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 14 01:16:28.632281 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Sep 14 01:16:28 2022] *** Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: nodename: dv-cinder-857075-4-iscsi Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: machine: x86_64 Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: clock source: unix Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: pcre jit disabled Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: detected number of CPU cores: 4 Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: current working directory: / Sep 14 01:16:28.634404 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: detected binary path: /usr/bin/uwsgi-core Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: your processes number limit is 47680 Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: your memory page size is 4096 bytes Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: detected max file descriptor number: 2048 Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: lock engine: pthread robust mutexes Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: thunder lock: enabled Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 14 01:16:28.635047 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 14 01:16:28.673335 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: Python main interpreter initialized at 0x5631d3d739c0 Sep 14 01:16:28.673335 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: python threads support enabled Sep 14 01:16:28.673335 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: your server socket listen backlog is limited to 100 connections Sep 14 01:16:28.673335 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: your mercy for graceful operations on workers is 90 seconds Sep 14 01:16:28.673696 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: mapped 403077 bytes (393 KB) for 2 cores Sep 14 01:16:28.673806 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: *** Operational MODE: preforking *** Sep 14 01:16:28.674099 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: *** uWSGI is running in multiple interpreter mode *** Sep 14 01:16:28.674099 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: spawned uWSGI master process (pid: 81401) Sep 14 01:16:28.674347 dv-cinder-857075-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 14 01:16:28.675443 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: spawned uWSGI worker 1 (pid: 81414, cores: 1) Sep 14 01:16:28.675443 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: spawned uWSGI worker 2 (pid: 81415, cores: 1) Sep 14 01:16:28.676757 dv-cinder-857075-4-iscsi devstack@keystone.service[81401]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 14 01:16:32.883363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:16:32.883363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: from cryptography.utils import int_from_bytes Sep 14 01:16:32.884215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:16:32.884215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: from cryptography.utils import int_from_bytes Sep 14 01:16:32.884648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:16:32.884648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: from cryptography.utils import int_from_bytes Sep 14 01:16:32.884752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 14 01:16:32.884752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: from cryptography.utils import int_from_bytes Sep 14 01:16:32.932471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 01:16:32.932643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 01:16:32.932794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] command line args: [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 01:16:32.932889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 14 01:16:32.933038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 14 01:16:32.933175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 01:16:32.933264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] command line args: [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 14 01:16:32.933344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ================================================================================ {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 01:16:32.933494 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 14 01:16:32.933580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] admin_token = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.933639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ================================================================================ {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 14 01:16:32.933710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] config_dir = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.933868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] admin_token = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.933964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] config_source = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] config_dir = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] debug = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] config_source = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] debug = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.934976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_config_append = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.935999 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936078 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_config_append = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_dir = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_file = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_dir = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_file = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.936826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.937897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938536 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.938979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.939943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] publish_errors = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940617 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940716 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] publish_errors = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940837 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.940996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] standard_threads = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.941755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.942261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.942408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] standard_threads = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.942586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.942753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.942753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943081 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] use_journal = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] use_json = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] use_journal = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] use_stderr = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] use_json = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.943985 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] use_syslog = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.944050 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] use_stderr = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.944050 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] use_syslog = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.944166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.944331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 14 01:16:32.944399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.944879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945391 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945694 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.945936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946448 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946601 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.946924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947080 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947161 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.947876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.948998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949135 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949845 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.949956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.950899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951009 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.951913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.952903 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953475 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953694 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.953803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954136 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.954891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.955980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.956916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957164 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957407 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957540 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.957890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958615 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.958930 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.959161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.959338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.959477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.959645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.959882 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960562 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.960889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.961972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.962913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.963953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964172 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.964889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965009 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.965917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.966967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967276 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.967919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968150 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.968938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969269 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969615 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.969967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.external = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.970910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.external = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971596 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.password = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.971947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] auth.token = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972058 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.password = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] auth.token = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972845 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.972956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.973897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974429 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.974891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975792 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.975903 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976839 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.976959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.977884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978588 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978700 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.978930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979267 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979501 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.979966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980189 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.980873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981299 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.981974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.982988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983108 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983227 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983227 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.983891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984124 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984124 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.984998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985455 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.985906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986157 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986505 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986624 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.986964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987751 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.987982 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988313 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.988989 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.989891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.990989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991880 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.991999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.992889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.993897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.994986 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.995995 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996340 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996792 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.996906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997136 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.997894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.998895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999227 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:32.999901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.000926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.001940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002243 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.002999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003303 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] role.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] role.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] role.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.003982 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] role.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004204 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.004998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.005970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006325 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.006910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.007892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008011 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008584 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.008917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.009932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010412 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010522 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010760 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.010981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011091 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.011995 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012550 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.012894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013226 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013344 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013679 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013789 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.013899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014039 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.014953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015307 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.connection = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015772 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.connection = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.015996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016448 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.016893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017235 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017456 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.017909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018170 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018742 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.018965 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019195 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81415) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 01:16:33.019857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 14 01:16:33.019857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81414) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 14 01:16:33.046402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.046627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.046733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.046870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.046993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.047115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.055093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.055462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.083175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.083314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.083468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.083594 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.136495 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.136753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.184248 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.184387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.184528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.184633 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.342980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.343102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.343174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.343300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.343380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.343544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.366512 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.366641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.366772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.366876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367594 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367756 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.367895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.378888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.379018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.391779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.391908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.392932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.393052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.393130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81415) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.393203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81414) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 14 01:16:33.409611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.409818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:16:33.410170 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.410296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.410423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:16:33.410525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:16:33.410651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.410730 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.410873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.410999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 14 01:16:33.411133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.411219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.411355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.411481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.411617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.411881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.420916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421188 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421711 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.421977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.422314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.428661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.429324 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.431579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.431860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.432412 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432493 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.432956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433039 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433469 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.433785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434346 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.434902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.456912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.457789 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.458387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.458665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.459300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.459580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.463291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.463574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.463852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.464229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.464516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.464796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.470123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.470403 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.471134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.471420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.475993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.476282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.476569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.476878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.477145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.477903 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.478051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.478187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.478599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.478691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.478992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.493260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.493535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.494482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.494762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.497357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.498642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.500586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.500874 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.501864 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.502179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.505047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.506482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.508302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.509674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.511513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.511808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.512088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.512370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.512655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.512940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.513133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.513233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.513508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.513791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.514090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.514382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.528584 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.528866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.529099 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.529382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.530317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.530603 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.530871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.531134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.536192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.537759 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.538029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.539303 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.539585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.539709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.541145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.541431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.542252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.542537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.542820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.543096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.543472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.544158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.544442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.544719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.544998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.545379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.554801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.555082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.555361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.555637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.556018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.556304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.556564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.556686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.556936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.557214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.557490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.557849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.559215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.559215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.573222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.574184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.576124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.577093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.579012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.580010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.580549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.581450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.582017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.582310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.582703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.582941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.583229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.583620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.589071 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.590067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.590547 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.591507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.593393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.593789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.594097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.594380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.594504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.594794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.595079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.595361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.602153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.602440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.602716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.603209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.603498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.603777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.609492 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81415) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.609769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.610075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.610345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.610628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.610753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81414) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 14 01:16:33.610891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.610992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611565 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.611950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612512 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612605 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.612926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.613023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81415) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.613103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.613341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.613640 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.613921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.614228 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81414) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 14 01:16:33.646680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5631d3d739c0 pid: 81415 (default app) Sep 14 01:16:33.648028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5631d3d739c0 pid: 81414 (default app) Sep 14 01:16:33.649908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3d07b-720a-4e8e-9fc1-516983a5035b None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:33.650215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3d07b-720a-4e8e-9fc1-516983a5035b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:33.650480 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af3d07b-720a-4e8e-9fc1-516983a5035b None None] PATH_INFO: `/v3` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:33.654609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Wed Sep 14 01:16:33 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:40.453773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.454638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.454638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.460712 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:40 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:40.466405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:40.466848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:40.466988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:40.537277 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: INFO dbcounter [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] Registered counter for database keystone Sep 14 01:16:40.556955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG oslo_db.sqlalchemy.engines [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81415) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 01:16:40.569047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writer thread running {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 01:16:40.754410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG passlib.handlers.bcrypt [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81415) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 01:16:40.754701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG passlib.handlers.bcrypt [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81415) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 01:16:40.813051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:40.814693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:40.968318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:40 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 503 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 14 01:16:41.453628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.531711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: INFO dbcounter [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] Registered counter for database keystone Sep 14 01:16:41.556515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG oslo_db.sqlalchemy.engines [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81414) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 14 01:16:41.568585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writer thread running {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 14 01:16:41.686547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45902d48-1c18-482e-8146-2eade02abe40 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.692356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f71bf19133e4a47a0d69d2812d255f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.692984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.693302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.693574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.704871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.763783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 01:16:41.764125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad69f1d9-e013-4a56-a427-738361d94927 admin admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.774344 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:16:41.780783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:41.799806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6409d29-35d8-46a8-870e-ed7ef7a79a2f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:41.805402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f71bf19133e4a47a0d69d2812d255f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:41.806556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:41.806835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:41.807101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] PATH_INFO: `/v3/projects/3f71bf19133e4a47a0d69d2812d255f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:41.808757 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] RBAC: Authorizing `identity:get_project(project_id=3f71bf19133e4a47a0d69d2812d255f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:41.861033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 14 01:16:41.861370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83bba5f-0af6-4fbf-8471-fb057627050e admin admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:41.866028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Sep 14 01:16:41 2022] GET /identity/v3/projects/3f71bf19133e4a47a0d69d2812d255f8 => generated 344 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:43.988314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fa59-15f1-452f-9872-10add62377d2 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:43.988873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fa59-15f1-452f-9872-10add62377d2 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:43.988873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621fa59-15f1-452f-9872-10add62377d2 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:43.994969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:44.004915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.005729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.005839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.032877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:44.034474 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:44.044406 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 14 01:16:44.309198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fa59-15f1-452f-9872-10add62377d2 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.323943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621fa59-15f1-452f-9872-10add62377d2 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.330624 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f71bf19133e4a47a0d69d2812d255f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.331766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.332379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.332722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.341738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.343639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbb1f64-22d6-4a81-8ee4-214211055235 admin admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.353053 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 14 01:16:44.359079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:44.368864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc85d3-97e3-4aa0-bfad-a511009a1f18 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:44.375131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3f71bf19133e4a47a0d69d2812d255f8', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:44.375762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:44.376087 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:44.376357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:44.377834 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:44.379729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46948d3-665c-4d6c-b953-6e98c084be7d admin admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:44.381327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Wed Sep 14 01:16:44 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:46.574047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866d362-199c-45ae-a829-c05a2a0c9e5a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:46.574623 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866d362-199c-45ae-a829-c05a2a0c9e5a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:46.574623 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d866d362-199c-45ae-a829-c05a2a0c9e5a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:46.575933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:46.581310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:46.581764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:46.581857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:46.600963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:46.602216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:46.624869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3c9e-95d4-437b-838f-1a3d0e94db20 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:46.625151 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3c9e-95d4-437b-838f-1a3d0e94db20 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:46.625421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c3c9e-95d4-437b-838f-1a3d0e94db20 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:46.626804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:16:46.631874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:46.632154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:46.632424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:46.674369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG passlib.handlers.bcrypt [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81414) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 14 01:16:46.674735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG passlib.handlers.bcrypt [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81414) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 14 01:16:46.687416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:46.723706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:46.725427 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:46.826215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:46.949953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:46.975428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.060380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9af620-a604-4732-a4cc-3f28c520fe71 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.083927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.097523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.098204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.098482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.098740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.109703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:16:47.118651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:16:47.119790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:16:47.120608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:16:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:16:47.125789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.184311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ac816-9a76-4fff-8e5d-352b50b950e2 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.186645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.205963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.223634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.224556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.224712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e89940-7c2f-49dd-8cdc-c11a1bf67751 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.224815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.225090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.257657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.258318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.258588 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.258855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.259524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:47.259816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.261213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.272769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:47.278669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.292681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.294361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.376154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd251819-b70d-4fab-872a-cb275c3bf6b5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.393079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 8/16] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:16:47.398887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.401521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.411112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.411947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.412222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.412490 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.413628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.415153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.416734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:47.422217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.436310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.495689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a79e0a-4403-4d13-935b-0924b6f0376f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.520777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83181c26-0406-41a6-a082-f80ce50e3fd0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.528541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.529435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.529706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.529971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.548091 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.549578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.554805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.555431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.555698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.555952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.557055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.558531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.560239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 10/18] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:47.565896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:16:47.567967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:16:47.568934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:16:47.569706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:16:47.569706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:16:47.569706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:16:47.569706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 9/19] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:16:47.574707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:47.629052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.645771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:47.666221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87adb75-bd1c-4a28-83b1-49ca2e8f70d5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.674418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90846a-eccf-4fbc-986d-68dcf55aef25 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:47.702607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.703222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.703499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.703771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.704510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.domain.id=default, user.id=f48f44237b4c4aa48da7b17eda7c3281) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:47.704816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.706297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7f095-1377-4c71-8565-e7aeeeea4766 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.709750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:47.710394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:47.710673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:47.710939 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:47.711580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:47.711878 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:47.712834 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 11/20] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/role_assignments?user.id=f48f44237b4c4aa48da7b17eda7c3281&scope.domain.id=default&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 233 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:47.713331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc11ca-76bf-447d-bdf8-c2688791d8b0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:47.733003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 10/21] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:16:49.974300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c84f7-f82d-4271-be30-caf5f6b5ac77 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:49.974817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c84f7-f82d-4271-be30-caf5f6b5ac77 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:49.974817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3c84f7-f82d-4271-be30-caf5f6b5ac77 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:49.976465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 12/22] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:49.981466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a7ec9-0ef3-48e1-8634-42a87e31315f None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:49.981871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a7ec9-0ef3-48e1-8634-42a87e31315f None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:49.981975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a7ec9-0ef3-48e1-8634-42a87e31315f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.007078 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-ba6a7ec9-0ef3-48e1-8634-42a87e31315f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:50.008635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-ba6a7ec9-0ef3-48e1-8634-42a87e31315f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:50.103531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 11/23] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:50.379112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1820-4734-4c35-8460-79fcbe461831 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.379787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1820-4734-4c35-8460-79fcbe461831 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.380422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c1820-4734-4c35-8460-79fcbe461831 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.382258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 13/24] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:50.387370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.387655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.387928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.415087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:50.416315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:50.469164 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:50.503586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 12/25] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:50.558556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1820-4734-4c35-8460-79fcbe461831 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.674761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c1820-4734-4c35-8460-79fcbe461831 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.721807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.722855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.723358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.723828 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.738980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:16:50.744803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:16:50.745724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:16:50.746493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 14/26] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:16:50.753570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.844739 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:50.850302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:50.885437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556bd323-64f2-433a-b4e0-f991d2943b3a None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:50.930807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:50.931358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.931636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.931896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.932569 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:50.932865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:50.937919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f99c8-4a60-4e5f-8c07-be2e716d1117 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:50.954492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 13/27] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:50.957835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6142-889a-448b-8aac-f100229872a2 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:50.958139 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6142-889a-448b-8aac-f100229872a2 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:50.958545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48c6142-889a-448b-8aac-f100229872a2 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:50.959748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 14/28] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:50.960432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c6142-889a-448b-8aac-f100229872a2 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.037870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.045637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd67bae9-805b-49e6-b5cb-b24c25e13665 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.098128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.099143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.099560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.100284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.116089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.118443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.122236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48c6142-889a-448b-8aac-f100229872a2 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-20d2116b-5fc3-4c65-aef5-6d05b486db92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:16:51.138219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:16:51.139120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:16:51.139925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:16:51.139925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:16:51.139925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:16:51.142078 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 15/29] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:16:51.143515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.143943 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.144373 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.169539 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:51.170832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:51.171646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.172533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.172992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.173489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.175535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.177268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085e666-8b2b-4b24-badd-732164ccb5e5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.179549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 15/30] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:16:50 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:51.185958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036eeb5-ff34-417a-be50-31527581d6c8 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.186519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036eeb5-ff34-417a-be50-31527581d6c8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.187011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7036eeb5-ff34-417a-be50-31527581d6c8 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.188514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 16/31] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:51 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:51.189440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036eeb5-ff34-417a-be50-31527581d6c8 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.232721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.254567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 16/32] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:51.255683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.313276 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7036eeb5-ff34-417a-be50-31527581d6c8 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.325538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.381511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.382478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.382948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.383375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.384189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:51.384855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.388290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab441b-28d8-4244-85af-199a78a235d6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.410354 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 17/33] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:51.411640 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.411929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.412199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.428544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544c5f0a-8825-47c3-8b2c-1389909bc58c None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.439609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:51.440755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:51.516372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.517028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.517304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.517573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.520090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 18/34] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:51.520838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.521624 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.523483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.529327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 17/35] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:51.535606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.635276 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.691068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:51.694633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce269a-8326-4d07-a6a6-be31df807095 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.697657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd7614-5622-48f4-9a5a-fccdfc7cbb47 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:51.734918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.735548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.735820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.736084 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.736738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.738334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff185e7-4912-4bbc-96a3-47bde034b6fb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.740707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:51.741608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.742077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.742513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] PATH_INFO: `/v3/domains/default/users/f48f44237b4c4aa48da7b17eda7c3281/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.745324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=f48f44237b4c4aa48da7b17eda7c3281, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:51.746966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:51.870175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 18/36] 10.222.0.26 () {58 vars in 1361 bytes} [Wed Sep 14 01:16:51 2022] PUT /identity/v3/domains/default/users/f48f44237b4c4aa48da7b17eda7c3281/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:16:51.871684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.883560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 19/37] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/projects => generated 301 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:51.884960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd926-ea45-4bf3-acc4-7cfe9d29db57 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:51.885247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd926-ea45-4bf3-acc4-7cfe9d29db57 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:51.885517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dbd926-ea45-4bf3-acc4-7cfe9d29db57 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:51.887090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 20/38] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:51.887897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd926-ea45-4bf3-acc4-7cfe9d29db57 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:51.997826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c16d2-6943-4e33-b7db-6357168e8f06 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.029518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.053672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dbd926-ea45-4bf3-acc4-7cfe9d29db57 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.062579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.063250 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.063527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.063771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.064499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.066696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da8ba7-8bb4-4ac1-8546-55e106738066 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.099741 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 19/39] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/roles => generated 225 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:52.101703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.101811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.102030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.127812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.128564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.128889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.129161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.129927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.132219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272f24ec-4536-4f45-a2e3-8abfa38ff0fc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.135464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:52.137372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.170216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 21/40] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 14 01:16:51 2022] POST /identity/v3/roles => generated 219 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:52.210162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aeb5a-e01d-492b-b896-8c8033e5a770 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.210747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aeb5a-e01d-492b-b896-8c8033e5a770 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.211026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563aeb5a-e01d-492b-b896-8c8033e5a770 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.212607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 22/41] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:16:52.223556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.223904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.224215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.245167 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 20/42] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:52.263390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:52.264961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:52.267694 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.371081 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 23/43] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:52.534414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:52.636389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c8d6c-1263-48f5-b64b-d45505a5c6e6 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.638719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.653184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:52.762796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bff88a-4f7b-41ba-9ba6-77ba54257b51 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.787214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.787785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.788067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.788340 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.789460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.791029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.792631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 21/44] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:52.798104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:52.799270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.800082 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.800528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.801123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.802567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.803983 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424e8e10-8998-4503-be23-0ac6f9aadd5e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.814209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:52.830024 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 24/45] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 14 01:16:52 2022] POST /identity/v3/roles => generated 223 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:16:52.914324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44583690-b4fa-4101-ac66-a75b19174240 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:52.951038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:52.951668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:52.951944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:52.952214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:52.952894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:52.954424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dee262-a667-4335-96c4-f36de8f76a22 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:52.986012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 22/46] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:16:52 2022] POST /identity/v3/projects => generated 312 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:54.272359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6737c-f271-4c41-8e36-9474451b2c89 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.272906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6737c-f271-4c41-8e36-9474451b2c89 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.272906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f6737c-f271-4c41-8e36-9474451b2c89 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.274297 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 25/47] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:54.279791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.280221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.280323 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.298829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:54.300023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:54.317731 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:54.370637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 23/48] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:16:54.632736 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6737c-f271-4c41-8e36-9474451b2c89 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.735602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f6737c-f271-4c41-8e36-9474451b2c89 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.769713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.770389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.770989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.771262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.783338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:16:54.789333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:16:54.790218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:16:54.790957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 26/49] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:16:54.796655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:54.896544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c052a7-61c2-4513-ac67-eb3fae5a13c5 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:54.932068 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:54.932675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:54.932951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:54.933216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:54.933888 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:54.934219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:54.935609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:54.944466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 24/50] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:54.950289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.011079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.058725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b9d990-cca7-4971-9ca4-c0ceb3d577d1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.099012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.099653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.099925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.100190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.101447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.102971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe5e804-15cd-452e-9184-bd283b955792 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.104593 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 27/51] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:16:54 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:55.113385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.127596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be74f9e-6754-47a7-84d8-a39726d5d1d3 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.127941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be74f9e-6754-47a7-84d8-a39726d5d1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.128217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be74f9e-6754-47a7-84d8-a39726d5d1d3 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.130275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 28/52] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:55.135485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.135779 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.136031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.139853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.155863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:55.157017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:55.222468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79559bdd-eae1-4255-a10d-cfb000b7e73d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.227294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 29/53] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:16:55.254561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.255241 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.255511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.255753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.256945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.258441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.260022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 25/54] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:55.265890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.434384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.478730 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd30539f-d02e-43b1-85a0-04656d0c3b4c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.490192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.513313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.513924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.514231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.514497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.515296 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.domain.id=default, user.id=f48f44237b4c4aa48da7b17eda7c3281) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:16:55.515515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.516953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.525450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 30/55] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Sep 14 01:16:55 2022] GET /identity/v3/role_assignments?user.id=f48f44237b4c4aa48da7b17eda7c3281&scope.domain.id=default&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 530 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:55.578555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.596634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1586e16f-1c4c-4084-94d5-c040d2c242f1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.632405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.633054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.633325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.633603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.634848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.636341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d68e227-7e59-4ddf-a6b2-adf4cb1ee9c4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.638033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 26/56] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:55.643996 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:55.757414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4c3dcb-a471-4e92-b8fd-8643e937e8a4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:55.777523 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:55.792102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:55.792708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:55.792985 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:55.793254 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:55.793936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:55.795415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d91818a-5aa1-4abb-860e-afc84bf5562f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:55.823328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 31/57] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Sep 14 01:16:55 2022] POST /identity/v3/projects => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:16:57.966925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9705-da7b-4e89-ba9c-97814bd28d2f None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:57.967751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9705-da7b-4e89-ba9c-97814bd28d2f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:57.967751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6f9705-da7b-4e89-ba9c-97814bd28d2f None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:57.969938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 27/58] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:16:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:16:57.976429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:57.977094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:57.977235 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:58.006001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:16:58.007217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:16:58.079127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 32/59] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:16:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:16:58.343138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f9705-da7b-4e89-ba9c-97814bd28d2f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:58.461804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6f9705-da7b-4e89-ba9c-97814bd28d2f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:58.495992 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:58.496601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:58.496872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:58.497138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:58.498321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:58.499777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311ee32-1168-4c47-ae30-b63a88a911af None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:58.501331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 28/60] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:16:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:16:58.510052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:16:58.615028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f502ba-4d12-4fd3-9372-fa0981d4b633 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:16:58.616596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:16:58.652281 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:16:58.652881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:16:58.653155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:16:58.653429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:16:58.654099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:16:58.655581 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6428d-1e87-46c0-9322-443f32f53857 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:16:58.688470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 33/61] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:16:58 2022] POST /identity/v3/users => generated 275 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:01.986453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71310ef-3096-47d3-937a-4d03e846843d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:01.986968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71310ef-3096-47d3-937a-4d03e846843d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:01.986968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71310ef-3096-47d3-937a-4d03e846843d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:01.988419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 29/62] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:01.993879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ddc2-7075-48da-95ab-110321e6c777 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:01.994355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ddc2-7075-48da-95ab-110321e6c777 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:01.994528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb7ddc2-7075-48da-95ab-110321e6c777 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.018871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-0eb7ddc2-7075-48da-95ab-110321e6c777 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:02.020074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-0eb7ddc2-7075-48da-95ab-110321e6c777 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:02.078513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49b3ea-c6ec-4c11-9ef0-1be0ae2d24a7 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.078819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49b3ea-c6ec-4c11-9ef0-1be0ae2d24a7 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.079078 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49b3ea-c6ec-4c11-9ef0-1be0ae2d24a7 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.086015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 30/63] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:02.088781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.089069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.089345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.118949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:02.120176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:02.148927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:02.162014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 34/64] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:02.176456 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2b684-a14a-4dbe-9271-b331b7802387 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.176781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2b684-a14a-4dbe-9271-b331b7802387 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.177061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a2b684-a14a-4dbe-9271-b331b7802387 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.178700 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 35/65] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:02.184020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.184528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.184991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.224282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:02.225503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:02.242351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 31/66] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:02.359153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 36/67] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:02.482843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:02.602630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:02.613692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f5a17a-f1d1-4fb7-95b2-2a9b350d6d97 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:02.646393 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:02.649924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:02.650592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.650867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.651131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.661750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cba7bd00-629b-4fb4-95f5-fefe5ea77834 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:02.666626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:02.667436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:02.668193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 32/68] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:02.675825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401216-bb64-4d5a-8544-29aba27edd84 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.676103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401216-bb64-4d5a-8544-29aba27edd84 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.676372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401216-bb64-4d5a-8544-29aba27edd84 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.677776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 33/69] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:02.678559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401216-bb64-4d5a-8544-29aba27edd84 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:02.691905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:02.741383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391cfcb-06b5-4639-8018-2a766e0fcca4 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:02.775951 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:02.776949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.777387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.777662 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.784156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401216-bb64-4d5a-8544-29aba27edd84 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:02.787789 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-8560e35c-85a0-4150-98e6-d3c0b5a5c8ff None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:02.793007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:02.794046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:02.794908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 37/70] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:02.796008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88645-df82-454d-ab46-6f81b2364b48 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.796008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88645-df82-454d-ab46-6f81b2364b48 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.796286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea88645-df82-454d-ab46-6f81b2364b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.813410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7ea88645-df82-454d-ab46-6f81b2364b48 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:02.814647 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7ea88645-df82-454d-ab46-6f81b2364b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:02.819963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:02.820573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.820846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.821112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.821770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:02.822100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:02.823477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:02.831379 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 34/71] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:02.831985 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:02.886266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 38/72] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:02.887592 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64defdeb-1ddc-4fd7-97ec-906072f00874 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.887864 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64defdeb-1ddc-4fd7-97ec-906072f00874 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.888128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64defdeb-1ddc-4fd7-97ec-906072f00874 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.889375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 39/73] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:02.893559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64defdeb-1ddc-4fd7-97ec-906072f00874 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:02.902746 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:02.903517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:02.935165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516673c7-837a-40c9-b405-bbdbd111672d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:02.970211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:02.971536 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:02.971815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:02.972086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:02.983222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:02.988450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:02.989863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 35/74] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:02.990653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.006086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64defdeb-1ddc-4fd7-97ec-906072f00874 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.042542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.043399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.043813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.044244 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.045048 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:03.045478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.047043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b814b19-f290-47e3-9000-1622e8911156 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.057868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 40/75] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.059272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.059550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.059816 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.080001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:03.081196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:03.111225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f4f47a-7bcf-44cf-8ccc-836f24984a4f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.145451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.148990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.149932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.150849 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.151344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.151890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.153238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.154173 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 41/76] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:03.155025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.156752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 36/77] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:02 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.157366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.160655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.260611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009cad0-e81e-45e0-9649-1b4089fb663f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.265894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7995b9e0-192d-4335-841d-6fb9578cc1f6 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.294896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.295531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.295879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.296072 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.302864 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.303485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.304202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.304474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.305149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:03.305472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.307372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.309224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.310692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.317118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 37/78] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.317118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.321102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 42/79] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.325549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.378104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.390979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.419196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff74ad-f779-4bb5-86c7-b5b398dc6e5b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.431059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66be94c-97c5-4b60-a7f4-02467b6f5cd3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.451734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.452362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.452635 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.452901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.454085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.455545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.457147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 38/80] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.457937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.469859 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.470511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.470826 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.471066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.537461 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.538965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.554362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d441ee9e-47cb-4106-9ed7-77aa87b41be1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.579572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.588748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.589356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.589729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.589963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.596438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.635156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 43/81] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:03.636016 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.661049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.662568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.732679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d69daa-0519-41eb-ac5d-04b7b3c8bfa6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.757091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 39/82] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:03.757942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.772111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.772750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.773026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.773296 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.774447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.775903 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.777646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 44/83] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:03.778270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.783676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.807946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:03.857768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90172c4b-206c-49ba-864f-4012a28a7872 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.878513 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfa0bd-64e0-4517-8900-05a9bc3da3cc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:03.890820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.891452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.891728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.892002 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.901961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:03.906899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:03.907800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:03.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 40/84] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:03.909839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:03.914670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:03.915290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:03.915567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:03.915879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:03.916519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:03.917999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:03.939691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 45/85] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:03 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:03.940527 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.013546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.031360 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4ff92-b062-4652-ae63-e04b971e7cb4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.039697 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.059726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691cef00-6bce-422f-9459-206e4298ae8c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.069535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.070215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.070493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.070762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.071992 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.073518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.075354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 41/86] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.076191 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.112011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.112774 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.113122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.113457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.114813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.116365 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.118232 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 46/87] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:03 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.118848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.189834 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99ad724-9446-4fdd-9fb8-a85b98879303 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.237284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.238330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.238606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.238877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.241041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671c544e-cddf-4672-91e0-469169fa278f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.272464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.286379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.286999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.287274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.287540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.288213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:04.288516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.289899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.301174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 47/88] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.303184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.327705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.336083 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.337569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.422406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982028d-9a65-4390-ad47-5e8e030bc51a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.460393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 42/89] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:04.461258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.464697 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.465511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.465931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.466625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.467553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=f48f44237b4c4aa48da7b17eda7c3281) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:04.468017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.469709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.476060 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 48/90] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/role_assignments?user.id=f48f44237b4c4aa48da7b17eda7c3281&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.477027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.574739 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.608229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.635710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13daf7ad-e6bf-4db4-a44c-9554876461c2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.639018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4232f1-d5d7-4146-b658-37c6e4013ae9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.677314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.677972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.678300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.678577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.679955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.681419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.683108 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 49/91] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.683900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.685589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.686484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.686954 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.687415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.688354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:04.688966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.691074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.697656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 43/92] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.698896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.829444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30216f9c-cc51-45bb-8750-bb206b8fcc62 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.869516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9554119b-3321-4750-b022-d8d89f896aa1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:04.870100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.870766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.871047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.871322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:04.874205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:04.874205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:04.875981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 50/93] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:04.882026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:04.910634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.918362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:04.947895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:04.948666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:04.948953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:04.949230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:05.082321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423ee273-4c03-4e13-a9a8-291073e7c21c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:05.101977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:05.103505 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:05.148305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:05.148938 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:05.149219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:05.149492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:05.150260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:05.150570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:05.152060 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:05.164201 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 51/94] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:05.258392 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 44/95] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:05.263090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:05.384008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:05.405646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f016e-fef2-4402-8380-dfb37f6f932b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:05.440328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:05.441747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:05.442393 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:05.442871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] PATH_INFO: `/v3/projects/6d5b0679a71143eda3f7e999c11d0d61` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:05.444318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5b0679a71143eda3f7e999c11d0d61)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:05.446067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4600b4bb-6d61-44f3-8abd-8545d3f7ff70 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:05.448315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 52/96] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:05 2022] GET /identity/v3/projects/6d5b0679a71143eda3f7e999c11d0d61 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:05.453945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:05.480519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:05.571176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d09dd8-4f3e-43d9-8a2e-d79e5835a4b5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:05.624028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:05.624713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:05.624998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:05.625273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:05.626032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=6d5b0679a71143eda3f7e999c11d0d61, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:05.626352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:05.627832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d6c9d3-8d6c-4743-9480-2f35ddd9054c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:05.634507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 45/97] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:05 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=6d5b0679a71143eda3f7e999c11d0d61&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:07.031349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71abbf-a0ff-4b4e-915f-cb86d2e7bb7a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.031870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71abbf-a0ff-4b4e-915f-cb86d2e7bb7a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.031975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e71abbf-a0ff-4b4e-915f-cb86d2e7bb7a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.033266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 53/98] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:07.038862 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.039263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.039476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.050152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ddd3-32ce-4658-8c5b-b60c91c1bd3d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.050432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ddd3-32ce-4658-8c5b-b60c91c1bd3d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.050704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84ddd3-32ce-4658-8c5b-b60c91c1bd3d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.052156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 54/99] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:07.057367 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.057646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.057918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.059208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:07.060432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:07.084094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:07.085275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:07.189567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 46/100] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:07.198559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 55/101] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:07.471289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:07.486210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:07.599286 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213f25ee-0448-432b-bc68-38b143ddfd98 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:07.649152 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:07.649797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.650102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.650372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.651957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:07.653414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:07.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 47/102] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:07.660935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:07.677879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:07.819614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b31f3-de4f-4699-a167-386b1f26121f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:07.822354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:07.839727 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:07.859650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:07.860383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.860724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.861058 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.872829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a49259ac-e330-4908-8137-99de4e47b2de None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:07.877938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:07.878980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:07.880067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:07.881564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 48/103] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:07.882976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9db75-f942-4df8-a6e6-ad85994620ed None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.883326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9db75-f942-4df8-a6e6-ad85994620ed None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.883661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e9db75-f942-4df8-a6e6-ad85994620ed None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.887608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 49/104] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:07.889355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9db75-f942-4df8-a6e6-ad85994620ed None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:07.903830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71270db-784f-482b-9afa-3c5325d7681e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:07.955963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:07.956861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.957241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.957608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:07.958794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:07.960705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d05e21-9ca4-4e87-9086-2f8a050b36a5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:07.996465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 56/105] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:17:07 2022] POST /identity/v3/projects => generated 302 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:07.998256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e14d8-d7d1-4456-b0b3-582770ae8c2c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:07.998861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e14d8-d7d1-4456-b0b3-582770ae8c2c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:07.999235 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e14d8-d7d1-4456-b0b3-582770ae8c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.008866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e9db75-f942-4df8-a6e6-ad85994620ed None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:08.023761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-e54e14d8-d7d1-4456-b0b3-582770ae8c2c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:08.025225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e54e14d8-d7d1-4456-b0b3-582770ae8c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:08.050673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:08.052248 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.052827 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.053282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.054252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:08.054807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:08.056758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26277bd-084d-46dc-bac4-4b08a2d6a259 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:08.068171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 50/106] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:08.070037 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e3c3-6d40-45e0-8080-ffede21a4592 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.070657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e3c3-6d40-45e0-8080-ffede21a4592 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.071199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0631e3c3-6d40-45e0-8080-ffede21a4592 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.073245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 51/107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:08.079234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631e3c3-6d40-45e0-8080-ffede21a4592 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.083147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.111004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 57/108] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:08.113193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.113693 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.114357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.143636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:08.146653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:08.252365 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.273077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0631e3c3-6d40-45e0-8080-ffede21a4592 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:08.274672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 58/109] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:08.310458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:08.311093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.311364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.311630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] PATH_INFO: `/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.325830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] RBAC: Authorizing `identity:get_user(user_id=a3910c7b8e9d4459ba2ed05a9a264a36)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:08.327247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:08.328917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 52/110] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:08.334660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.437218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d240808-b8de-4669-95bd-21703c5577b9 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:08.472766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:08.473558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.473839 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.474146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.511342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.539867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.552037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:08.553537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297613c-f3fe-4d8a-81e1-eb884e175715 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:08.658028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.661656 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 59/111] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:08.662969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2af1-3a13-4e3c-bac7-570a223989c6 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.663256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2af1-3a13-4e3c-bac7-570a223989c6 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.663546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a2af1-3a13-4e3c-bac7-570a223989c6 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.664921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 60/112] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:08.670897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a2af1-3a13-4e3c-bac7-570a223989c6 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.708272 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8340b12-2c02-47fa-82df-a737d4e0705f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:08.749125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:08.750364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.751082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.751648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.766314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:08.773529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-eece37c0-308c-4efa-bfbb-8281c96832e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:08.773811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:08.774814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:08.775571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:08.779386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 53/113] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:08.784658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.784658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.784658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.794199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a2af1-3a13-4e3c-bac7-570a223989c6 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:08.816606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:08.818526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:08.830862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:08.831496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:08.831776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:08.832046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:08.833224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:08.834769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:08.836423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 61/114] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:08.837191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.914699 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 54/115] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:08.916252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:08.980812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:08.991601 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3970e567-a08d-4161-a133-0bbaad9cbf99 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.044889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.045528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.045803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.046099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.059306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5952e0-a9af-45ca-87d6-fcfd43196251 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.066771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:09.075699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:09.076516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:09.077290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 62/116] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:09.077900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.103169 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.124063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.124684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.124963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.125235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.126536 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:09.126846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.128727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.139569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 55/117] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.140700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.204827 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12051cc2-30ae-4148-8195-ebd07635652e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.246317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.247411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.248201 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.248911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.249831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=a3910c7b8e9d4459ba2ed05a9a264a36) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:09.250301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.251912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8713898-7e71-4841-95ff-97ca519de2fd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.253455 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffab272-ddfd-459d-97a1-458a8a050fdf None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.259118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 63/118] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/role_assignments?user.id=a3910c7b8e9d4459ba2ed05a9a264a36&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.260673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdfa76-c4f9-430e-b868-0e5c756feaae None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.261569 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdfa76-c4f9-430e-b868-0e5c756feaae None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.262092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fdfa76-c4f9-430e-b868-0e5c756feaae None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.264186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 64/119] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:09.265244 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdfa76-c4f9-430e-b868-0e5c756feaae None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.272526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.298995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.299632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.300055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.300157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.300846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:09.301140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.302758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1769ac5-520d-4f03-95db-469a4b67d52e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.312263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 56/120] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.313540 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.313836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.314154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.334990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:09.336296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:09.387229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.409672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fdfa76-c4f9-430e-b868-0e5c756feaae None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.427212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 57/121] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:09.428538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.466786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.467840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.468188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.468532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.469963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.471715 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.478092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 65/122] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.478092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.559560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8e7c6-f7bc-4743-82b4-7c282a51c0e2 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.568077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.598018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83399178-62fe-4790-b0c7-ed6925626e21 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.631898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.632641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.632920 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.633189 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.638020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.638930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.639411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.639897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.641337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.643338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.645222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 66/123] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.646448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.651256 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:09.668182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:09.669138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:09.670088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:09.670854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 58/124] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:09.671923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.762226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2cf3f5-79cb-4ef6-828a-339e6710e075 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.803666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.804334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.804609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.804874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.806027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.807524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.809117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 67/125] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.809876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.883368 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3117284-87d7-4f24-8453-84261c153f3c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.908814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.921683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.922600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.923055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.923460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.924765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.926311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035b0567-c26d-4928-9b03-a2fb6b2b81a9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:09.926609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.928453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 59/126] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.929334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:09.961644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:09.973550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:09.974981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:09.975431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:09.976342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:09.977466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:09.977941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:09.980524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:09.992713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 68/127] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:09.993725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.037715 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3164573-5a98-48be-956e-a44953a0a7c7 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.077750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.079021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.079464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.079900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/f48f44237b4c4aa48da7b17eda7c3281/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.081977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, user_id=f48f44237b4c4aa48da7b17eda7c3281, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.083843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.099396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 60/128] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:09 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/f48f44237b4c4aa48da7b17eda7c3281/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:10.100590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.145747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c706-12b6-4bda-bb6f-0f5814f2e8fd None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.204542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:10.210183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.210850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.211128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.211394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.216460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544551a8-9999-4006-accc-cd71e56f9d94 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.239421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:10.245061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:10.245912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:10.246721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 69/129] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:10.255291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.263361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.264590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.265304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.265771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/3341ca87c68d41e8a973c93cb5345109/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.272616 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, user_id=3341ca87c68d41e8a973c93cb5345109, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.274497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.281522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:10.317190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 61/130] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:10 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/3341ca87c68d41e8a973c93cb5345109/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:10.318101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.446546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88385db-98eb-4d86-8502-29e9dbb6de24 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.526147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.527221 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.527723 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.528184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.529121 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e3ddc2-736b-464b-bb73-6db7f2e5b381 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.529908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.531929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.535443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 70/131] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:10 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:10.539658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.603051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.603600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.603928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.604202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.604897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:10.605246 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.610038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.619116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 62/132] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:10.625552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.644359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:10.700736 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:10.736182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc5cfe1-bf67-4351-8bec-d544252112f7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.762674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195322-a547-4da5-8298-a30660684a2f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.774425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.775069 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.775349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.775623 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.776819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.778561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a34057-fbf5-4588-89de-155dd9c4d0a0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.780260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 71/133] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:10 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:10.781491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd250-ccf6-4eb1-8f20-71141e9d9b74 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.781772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd250-ccf6-4eb1-8f20-71141e9d9b74 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.782133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067bd250-ccf6-4eb1-8f20-71141e9d9b74 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.787486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 72/134] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:10.788305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bd250-ccf6-4eb1-8f20-71141e9d9b74 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.809485 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:10.810329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.810706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.811080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.812347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:10.814750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6dd4e1-61d7-4b87-9381-ed955412ba7c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:10.818042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 63/135] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:10 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:10.819416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:10.820165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:10.820672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:10.843689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:10.845476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:10.934439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067bd250-ccf6-4eb1-8f20-71141e9d9b74 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:10.961476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 64/136] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:10.962399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:10.990662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.000669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.001900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.005059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.005330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/3341ca87c68d41e8a973c93cb5345109/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.007766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, user_id=3341ca87c68d41e8a973c93cb5345109, role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.009233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.040018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 73/137] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:10 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/3341ca87c68d41e8a973c93cb5345109/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.124608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b320993-5c03-47ff-8063-ac0198adf710 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.164458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.165106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.165378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.165648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] PATH_INFO: `/v3/projects/6d5b0679a71143eda3f7e999c11d0d61` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.166799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5b0679a71143eda3f7e999c11d0d61)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.168247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.169907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 65/138] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:10 2022] GET /identity/v3/projects/6d5b0679a71143eda3f7e999c11d0d61 => generated 312 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:11.176209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.180441 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.291547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d69644c-f104-4ab7-9e20-ffb0f223891c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.328538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.329303 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.329635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.329914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] PATH_INFO: `/v3/projects/6d5b0679a71143eda3f7e999c11d0d61/users/3341ca87c68d41e8a973c93cb5345109/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.331973 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6d5b0679a71143eda3f7e999c11d0d61, user_id=3341ca87c68d41e8a973c93cb5345109, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.333501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.353781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.362155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 74/139] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:11 2022] PUT /identity/v3/projects/6d5b0679a71143eda3f7e999c11d0d61/users/3341ca87c68d41e8a973c93cb5345109/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:11.453330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.476488 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656abd4-d58b-4dcc-be76-7041f39c4d38 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.513341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.514133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.514630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.514977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.516362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.518144 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd215a3c-ca73-4057-8f3e-a598029e50d8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.519979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 66/140] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:11.526237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:11.619795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:11.662515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e24e879-eaae-46c1-904d-dcebe6192ba8 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:11.697464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:11.698114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:11.698387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:11.698648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:11.699317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:11.700761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdee00ce-3e67-449b-80ca-2e817f3dc7c8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:11.726793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 75/141] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:11 2022] POST /identity/v3/users => generated 283 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:12.239324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c124fa5-7df3-439a-a705-3565d808d997 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.239825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c124fa5-7df3-439a-a705-3565d808d997 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.239926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c124fa5-7df3-439a-a705-3565d808d997 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.241399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 67/142] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:12.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.247539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.248003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.267290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:12.268818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:12.345895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 76/143] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:12.618385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c124fa5-7df3-439a-a705-3565d808d997 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.739007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c124fa5-7df3-439a-a705-3565d808d997 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.779543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.780196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.780475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.780752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.796969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:12.802153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:12.802980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:12.803738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 68/144] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:12.810654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:12.888468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:12.944828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf9634-8180-425e-a208-93ba8e6540c7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:12.991784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:12.992408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:12.992679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:12.992946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:12.993613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:12.993909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:12.995333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.008412 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 77/145] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:13.014172 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.036129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:13.160453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b3e534-634e-44a1-9b5e-788a76e3c8a5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.204074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.204720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.204998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.205282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] PATH_INFO: `/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.206589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] RBAC: Authorizing `identity:get_user(user_id=a3910c7b8e9d4459ba2ed05a9a264a36)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.208132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.210120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 69/146] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:13 2022] GET /identity/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:13.215352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.366192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addb59e9-2f69-4c93-9fff-406e00ebcec4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.379632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:13.401952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.402621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.402900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.403168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.404300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.405777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead17859-ed77-41db-8112-f12646dcc600 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.407634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 78/147] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:13 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:13.413331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:13.461209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c257f-a3d9-451a-97f2-19e1a0732eeb None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.461540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c257f-a3d9-451a-97f2-19e1a0732eeb None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.461761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5c257f-a3d9-451a-97f2-19e1a0732eeb None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.463749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 79/148] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:13.469534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.469815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.470117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.496123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:13.497333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:13.518385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:13.554014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2fda4-3b38-4f6c-a8c7-3d774bb9b0f7 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:13.615867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:13.616906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:13.617834 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:13.618338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/a3910c7b8e9d4459ba2ed05a9a264a36/roles/e863c8d690274edbabdf68d32b2c88fe` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:13.620700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, user_id=a3910c7b8e9d4459ba2ed05a9a264a36, role_id=e863c8d690274edbabdf68d32b2c88fe)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:13.622581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c645f-c3ce-4597-bb57-36141638592a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:13.649213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 80/149] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:13.652887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 70/150] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:13 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/users/a3910c7b8e9d4459ba2ed05a9a264a36/roles/e863c8d690274edbabdf68d32b2c88fe => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:14.289846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:14.415836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d28f-0269-4502-afc0-602f9f56dbf6 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.416366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d28f-0269-4502-afc0-602f9f56dbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.416366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2392d28f-0269-4502-afc0-602f9f56dbf6 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.417787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 71/151] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:14.430845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.431190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.431377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.433974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2782c0c-40f8-496a-b0f9-ba6dff051841 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:14.471781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:14.473006 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:14.474364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:14.515022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:14.515667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.515941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.516207 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.532633 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:14.537643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:14.538546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:14.539298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:14.539866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 81/152] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:14.550964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:14.612590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 72/153] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 14 01:17:14.657666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7af78d-7792-4621-a84e-517a5e16ac25 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:14.692626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:14.693583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.694085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.694540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.695386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:14.695888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:14.697450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db577244-eddc-4a7a-9cca-39ef0efbe3ec None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:14.706790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 82/154] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:14.712816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:14.728370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8d08e-6bd0-49e8-b45e-46d2f67f70f7 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.729014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8d08e-6bd0-49e8-b45e-46d2f67f70f7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.729535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f8d08e-6bd0-49e8-b45e-46d2f67f70f7 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.731095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 83/155] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:14.736304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.736813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.737325 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.757190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:14.758737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:14.828860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:14.838531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 84/156] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:14.878508 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:14.927441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a68c83a-55d6-4212-8138-14aa3a9c4b1e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:14.965158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:14.965719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:14.966015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:14.966424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:14.967686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:14.969253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:14.970914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 73/157] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:14 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:14.976771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.092368 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823add34-4cef-4325-ae96-a34c27f7332e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:15.142758 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:15.143391 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.143674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.143950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.179579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:15.227314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:15.228790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:15.265229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.382640 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 85/158] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:15.388871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.443327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49494a3c-2c43-46ac-8337-ea8e3d58ae65 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:15.512961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:15.518686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:15.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.519607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.519876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.522129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:15.546913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:15.551898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:15.554322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 74/159] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:15.556333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.596128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a3c95-3eb4-4846-9e99-c0c911bca0b1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:15.652803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:15.653467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.653748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.654040 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.655245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:15.656722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:15.658529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 86/160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:15.659386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.740439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e9a46-b072-4745-82ed-9e362b8d844b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:15.778334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:15.778898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.779171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.779437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.790529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-dcca8268-4983-4731-8c9a-5f2d5c3c399f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:15.795399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:15.796358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:15.797118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 75/161] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:15.797808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2242-eb43-406b-8420-12ad814ab923 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.798391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2242-eb43-406b-8420-12ad814ab923 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.798391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aa2242-eb43-406b-8420-12ad814ab923 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.800067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 76/162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:15.800775 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa2242-eb43-406b-8420-12ad814ab923 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.856192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0ad6c3-19f3-4666-af7c-d57b37948184 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:15.861815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:15.893505 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:15.903176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:15.903809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:15.904088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:15.904407 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:15.905100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:15.905396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:15.907035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:15.917111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 87/163] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:15.918544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:15.950691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aa2242-eb43-406b-8420-12ad814ab923 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.016007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.016625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.016896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.017159 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.017890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:16.018216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.019689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1552c2e-4f37-433c-98ee-40814abd8d35 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:16.034946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 77/164] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 608 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:16.036182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.036463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.036734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.073043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30f86ac-b8b3-42a0-bc6a-1d9979de1c84 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.073370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:16.076377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:16.112982 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.114115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.114641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.115320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.116575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:16.117058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.118723 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:16.128297 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 88/165] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:16.129883 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:16.213543 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:16.251406 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 78/166] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:16.252335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:16.297484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768bf3d3-5860-4a0f-ac21-dbc4f6c54219 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.355928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:16.357865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.358542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.359076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.359177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] PATH_INFO: `/v3/users/f48f44237b4c4aa48da7b17eda7c3281` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.360313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] RBAC: Authorizing `identity:get_user(user_id=f48f44237b4c4aa48da7b17eda7c3281)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.361809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:16.363619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 89/167] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:16 2022] GET /identity/v3/users/f48f44237b4c4aa48da7b17eda7c3281 => generated 247 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:16.371708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:16.481079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12f233-a526-404f-adf4-ee1afae86151 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.526238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e516c469-da7d-4357-9417-bac240f35afe None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.539022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.539676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.539948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.540215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.543483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.544951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:16.546610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 79/168] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:16 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:16.552484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:16.575421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:16.599921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.600545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.600819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.601080 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.713185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416219b9-1419-47c0-88f6-0d5a1904cc98 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:16.738685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:16.760260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.761753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:16.770643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:16.771268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:16.771540 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:16.771806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:16.911162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:16.912706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b01e2b-0066-4b8f-8e8c-81035e727cf4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.043158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 90/169] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 672 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:17.044576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.050610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 80/170] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:17:17.051698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f73eb-2dd3-42ae-a556-94cb5eb59ffb None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.057131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f73eb-2dd3-42ae-a556-94cb5eb59ffb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.057453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7f73eb-2dd3-42ae-a556-94cb5eb59ffb None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.059107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 81/171] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:17.059724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f73eb-2dd3-42ae-a556-94cb5eb59ffb None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.099468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.102713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.229607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7f73eb-2dd3-42ae-a556-94cb5eb59ffb None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.229962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1017c912-b7ee-4805-a76f-579913665c75 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.274678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.275105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.275344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.275616 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.275795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.275894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.276038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.276317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.276977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.278814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.281182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 82/172] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.282134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.293797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-c7313789-c7d0-4934-b554-cb4818732f95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:17.300331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:17.301218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:17.302065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 91/173] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:17.306023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78d44-c8dc-45d7-b37f-b43b8730b3d2 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.306023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78d44-c8dc-45d7-b37f-b43b8730b3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.306023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d78d44-c8dc-45d7-b37f-b43b8730b3d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.326191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-33d78d44-c8dc-45d7-b37f-b43b8730b3d2 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:17.327546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-33d78d44-c8dc-45d7-b37f-b43b8730b3d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:17.376965 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.394132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c935994-21eb-4bbf-80c3-6266441279e3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.408596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 92/174] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:17.415686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcbdd-1843-499c-8202-b46b47caa3ae None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.416222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcbdd-1843-499c-8202-b46b47caa3ae None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.416720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bcbdd-1843-499c-8202-b46b47caa3ae None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.418447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 93/175] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:17.419338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bcbdd-1843-499c-8202-b46b47caa3ae None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.442721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.443370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.443628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.443897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] PATH_INFO: `/v3/projects/6d5b0679a71143eda3f7e999c11d0d61` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.445976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.447317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] RBAC: Authorizing `identity:get_project(project_id=6d5b0679a71143eda3f7e999c11d0d61)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.448793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.458632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 83/176] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/projects/6d5b0679a71143eda3f7e999c11d0d61 => generated 312 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.459495 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.566836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bcbdd-1843-499c-8202-b46b47caa3ae None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.588999 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c28405-3850-417f-802d-3484d9e6f1e5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.611535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.612470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.612936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.613398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.614307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=f48f44237b4c4aa48da7b17eda7c3281) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:17.614613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.616066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6812e77-744a-4c12-871b-6fa91e77bdaf None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.626102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 94/177] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/role_assignments?user.id=f48f44237b4c4aa48da7b17eda7c3281&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.626102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643ef19-6c3d-4f07-aa38-0c8894f8cf03 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.626102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643ef19-6c3d-4f07-aa38-0c8894f8cf03 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.626102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643ef19-6c3d-4f07-aa38-0c8894f8cf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.631017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.631980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.632416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.632858 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.633546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:17.633850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.635346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.645517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d643ef19-6c3d-4f07-aa38-0c8894f8cf03 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:17.646477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 84/178] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.647476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.647962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d643ef19-6c3d-4f07-aa38-0c8894f8cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:17.665068 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.732913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 95/179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:17.734094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba70ce-96f3-4032-ac87-f4262a5021f5 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.734378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba70ce-96f3-4032-ac87-f4262a5021f5 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.734646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba70ce-96f3-4032-ac87-f4262a5021f5 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.735962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 96/180] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:17.737042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835f36d-72c6-4bcb-955c-b82aaf98072b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.737314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835f36d-72c6-4bcb-955c-b82aaf98072b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.737577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835f36d-72c6-4bcb-955c-b82aaf98072b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.739044 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 97/181] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:17.739591 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835f36d-72c6-4bcb-955c-b82aaf98072b None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.752596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.786485 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c672e2-5dd9-4ecc-8e8d-72069698cac2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.829685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.829685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.829929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.830164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.831005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=6d5b0679a71143eda3f7e999c11d0d61, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:17.831335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.832931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.840962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 85/182] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=6d5b0679a71143eda3f7e999c11d0d61&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.841276 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:17.876073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835f36d-72c6-4bcb-955c-b82aaf98072b None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:17.928148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:17.928802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.931257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.931538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] PATH_INFO: `/v3/users/3341ca87c68d41e8a973c93cb5345109` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.933236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] RBAC: Authorizing `identity:get_user(user_id=3341ca87c68d41e8a973c93cb5345109)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:17.934346 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9611c5c-c785-4c99-a43f-e19a706f967a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:17.936237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 98/183] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/users/3341ca87c68d41e8a973c93cb5345109 => generated 275 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:17.940309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d39d-3be1-4b85-8889-1990e9e35cee None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:17.940309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d39d-3be1-4b85-8889-1990e9e35cee None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:17.940309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2176d39d-3be1-4b85-8889-1990e9e35cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:17.963481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:17.964082 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2176d39d-3be1-4b85-8889-1990e9e35cee None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:17.965415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2176d39d-3be1-4b85-8889-1990e9e35cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:17.994074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a606b6-1904-42c6-8163-ae75fdb618b0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.041909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.042619 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.042897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.043155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.055073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:18.060946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:18.061843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:18.062720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:18.063320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 99/184] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:18.063389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 86/185] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:18.063452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.063515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.063586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.063586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.087345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.100745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:18.102449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:18.182198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 100/186] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:18.187911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.199316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be5ec53-b6ae-49f7-8a6f-34acb846ea36 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.253015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.253633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.253907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.254209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.266382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.311912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b60729-7ab5-4632-a0e6-e2169a0dbe5d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.336820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.338452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.354187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.354824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.355105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.355360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.357268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.367378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:18.372252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:18.373238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 101/187] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:18.379241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.449892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 87/188] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:18.450838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.480309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92bf1b-98b4-42b5-b37e-435cbf79633b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.490420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.514122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.514724 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.514997 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.515247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.515904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:18.516203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.517602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.527005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 102/189] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:18.527005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.552004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21606f7d-d73d-485f-a7cf-d76758a27c3c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.582989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.585141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.585840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.586138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.586424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.587522 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.589036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.590579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 88/190] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:18.595079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.628084 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe4667-8830-4b9e-a9a7-4661a29b2214 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.661810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.662444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.662718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.662991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.663651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:18.663949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.665329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.673425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 103/191] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:18.675269 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.692159 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87c5d6-67ff-4865-a12d-b9cc94f1650b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.699365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.726286 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.726927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.727201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.727470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.728597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.730146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.731811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 89/192] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:18.732568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.775748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f9373-0464-4183-a9d6-12be68af5471 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.792824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.809788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.810445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.810722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.810977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.821254 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:18.826501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:18.827491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:18.828306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 104/193] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:18.829003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.832784 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae0b546-671a-4741-82dc-f75af12a85d3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.866981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.867612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.868236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.868236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.881139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.882672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.884295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 90/194] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:18.885053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:18.913667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:18.930904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f201db8-473d-4721-aafd-57658c4870fc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.964706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:18.965318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:18.965578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:18.965848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:18.966546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:18.968064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:18.981032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cf3f70-c173-41a2-9275-cf17659ca666 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:18.988277 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 105/195] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:18 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:18.988277 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.016985 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.017751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.017921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.018242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.019514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.021067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.022796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 91/196] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:19.027819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.036695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.124892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2091c-37ed-49dc-afd6-9098c7ca5973 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.137806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.141835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc68200-716a-42c2-9a90-7ef27e9bbe2c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.162642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825a95-f900-4083-83b4-9df497076993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.163086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825a95-f900-4083-83b4-9df497076993 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.163360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825a95-f900-4083-83b4-9df497076993 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.164222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67825a95-f900-4083-83b4-9df497076993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.164337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67825a95-f900-4083-83b4-9df497076993 None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=3341ca87c68d41e8a973c93cb5345109) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:19.164638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67825a95-f900-4083-83b4-9df497076993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.166122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67825a95-f900-4083-83b4-9df497076993 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.172538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 106/197] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:18 2022] GET /identity/v3/role_assignments?user.id=3341ca87c68d41e8a973c93cb5345109&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:19.173128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825a95-f900-4083-83b4-9df497076993 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.195261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.196274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.196549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.196814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.197507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:19.197809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.199292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.210830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 92/198] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:19.210830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.308367 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.314852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67825a95-f900-4083-83b4-9df497076993 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.355349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427910da-896d-4fbb-89f6-8832fe40440f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.362499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.363142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.363408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.363777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.409805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.410390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.410724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.410987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.445246 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.464222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.466337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.538067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.538692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.588340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.602464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 107/199] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:19.604282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.706980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 93/200] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:19.708201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.786679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d0db4-ecf1-4a6d-b62e-f24c97035d1c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.827223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:19.850373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.851020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.852095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.852095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] PATH_INFO: `/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.854695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] RBAC: Authorizing `identity:get_user(user_id=a3910c7b8e9d4459ba2ed05a9a264a36)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.856161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.858118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 108/201] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/users/a3910c7b8e9d4459ba2ed05a9a264a36 => generated 289 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:19.858664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:19.896321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade14c1-4e55-438d-8bc3-3998c5d5ee21 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:19.934725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:19.938014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:19.938014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:19.938014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:19.938014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:19.938503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:19.940172 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 94/202] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:19.941293 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.013704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.046842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9773f-5fb6-4142-a774-690c73fa4a74 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.057191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d99f849-a93f-4b33-90e3-2f261412e4dd None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.076731 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.083891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.084720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.085154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.085575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.102434 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.103083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.103362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.103645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.104768 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.106768 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.113487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 109/203] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:20.113487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.175051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.176719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.236218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6cc35-fd23-4ab2-ba1c-e3293c4631b6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.275485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.276351 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.276768 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.277198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.278194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=300ad085fe6143ad9ab831560faeacf8) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:20.278836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.281208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.283764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.290973 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 110/204] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/role_assignments?user.id=300ad085fe6143ad9ab831560faeacf8&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:20.292119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.292487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 95/205] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:20.307987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.371922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.413702 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208ee4d-cc02-4e1a-83fc-cebfa242ba94 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.454541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.455490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.455967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.456423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.457511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=300ad085fe6143ad9ab831560faeacf8) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:20.458069 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.460083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.466821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 111/206] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/role_assignments?user.id=300ad085fe6143ad9ab831560faeacf8&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:20.507716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f6c300-afe0-4547-9bd0-202f2e33b5ba None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.544192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.544793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.545075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.545345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.546625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.548089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0aefc-7296-49a2-8cb2-a555f034e33c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.549839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 96/207] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:20.556328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:20.622247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:20.679749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c4c150-3948-4f3b-ac27-fd8f1b55bbcb None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:20.729611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:20.730704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:20.731253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:20.731712 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:20.732648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5, user.id=a3910c7b8e9d4459ba2ed05a9a264a36) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:20.733165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:20.734852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486e28e-cea7-4611-a1bd-45b287d1acd8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:20.741702 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 112/208] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:20 2022] GET /identity/v3/role_assignments?user.id=a3910c7b8e9d4459ba2ed05a9a264a36&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:21.478265 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc611-e719-44d2-bb6d-5370f259f2f3 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:21.478795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc611-e719-44d2-bb6d-5370f259f2f3 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:21.478795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edc611-e719-44d2-bb6d-5370f259f2f3 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:21.480404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 97/209] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:21.485847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440748-a88d-44cf-b5b9-a8cdc9388970 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:21.486175 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440748-a88d-44cf-b5b9-a8cdc9388970 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:21.486450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67440748-a88d-44cf-b5b9-a8cdc9388970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:21.507094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-67440748-a88d-44cf-b5b9-a8cdc9388970 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:21.508298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-67440748-a88d-44cf-b5b9-a8cdc9388970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:21.578641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 113/210] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:21.683114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0e0-2a71-43c7-a7d8-c4dd8a7fd969 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:21.683464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0e0-2a71-43c7-a7d8-c4dd8a7fd969 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:21.683656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d1c0e0-2a71-43c7-a7d8-c4dd8a7fd969 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:21.685281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 98/211] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:21.690823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:21.691260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:21.691419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:21.710742 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:21.711968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:21.763644 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:21.781375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 114/212] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:21.846860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c0e0-2a71-43c7-a7d8-c4dd8a7fd969 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:21.943921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:21.948813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d1c0e0-2a71-43c7-a7d8-c4dd8a7fd969 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:21.984405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:21.985038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:21.985308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:21.985572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:21.999805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:22.005239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:22.006125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:22.006959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:22.009626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 99/213] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:22.015234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.064776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.124463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96b3b5-3b26-45f4-8270-727d37175702 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.159510 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.160516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.161122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.161588 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.162461 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:22.162945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.164520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.176370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 115/214] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:22.182915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.192207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02e8dea-e12a-4af3-93ea-e1dc30c33fd2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.228111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.228725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.228993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.229302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.230745 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.232215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.238449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 100/215] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:22.240080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.248844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:22.274623 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:22.313036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff05261-a257-4171-8db6-fc52a4eadb7f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.352061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.353073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.353857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.354705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] PATH_INFO: `/v3/users/42e846222023460ea0ec229b9f697d85` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.357261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3dcde4-ec1e-4f31-962e-ec13f5aabc82 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.371486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] RBAC: Authorizing `identity:get_user(user_id=42e846222023460ea0ec229b9f697d85)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.374238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.381282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 116/216] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/users/42e846222023460ea0ec229b9f697d85 => generated 297 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:22.382668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.394056 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.394918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.395258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.395638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.396458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.398153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f501194f-e8d7-4967-a1a3-cb70268cce79 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.433043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 101/217] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:22 2022] POST /identity/v3/users => generated 297 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:22.528453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f43a69-bcb4-4fbf-8741-5671c141b7f4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.537941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:22.569223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.570250 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.570721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.571203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.648450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.650429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.756887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae0bf-3166-4207-9c39-ccac90b31e7d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.757401 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 117/218] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:22.758440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae0bf-3166-4207-9c39-ccac90b31e7d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.758899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ae0bf-3166-4207-9c39-ccac90b31e7d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.765441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 102/219] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:22.765583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.770449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4991f-344f-4308-85e1-07c82142f815 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.771022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4991f-344f-4308-85e1-07c82142f815 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.771511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b4991f-344f-4308-85e1-07c82142f815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.773232 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:22.797768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-b9b4991f-344f-4308-85e1-07c82142f815 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:22.799325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b9b4991f-344f-4308-85e1-07c82142f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:22.898272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4dbc1-5b31-4491-8fef-fc324cc0c18e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:22.905042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 103/220] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:22.938442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:22.939094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.939366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.939635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.943193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:22.944661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:22.946739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 118/221] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:22.947270 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9c9d-6df6-422f-adbe-039b196321db None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.947810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9c9d-6df6-422f-adbe-039b196321db None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.948329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dd9c9d-6df6-422f-adbe-039b196321db None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.951067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 104/222] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:22.952210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:22.958825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:22.959211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:22.959554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:22.985676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:22.987694 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:23.002362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:23.072493 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:23.120122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35e56b-6bcb-435f-a727-bd6460b3762b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.123936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 105/223] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:23.157663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.158390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.158675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.158944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.159739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=42e846222023460ea0ec229b9f697d85) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:23.160047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:23.161611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:23.167653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 119/224] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:22 2022] GET /identity/v3/role_assignments?user.id=42e846222023460ea0ec229b9f697d85&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:23.218467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.356466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca610678-ec82-4314-a927-d4817ef85afc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.393130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.393771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.394102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.394370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.404965 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:23.410034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:23.411017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:23.412355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 106/225] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:23.417506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.419290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.422964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:23.538380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a643c-db5d-496d-86e3-c78d8e4295d8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.595867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99822b96-a53e-4dab-9a91-75d8f45595ce None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.611327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.611984 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.612262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.612515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.634293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.634998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:23.635348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.635763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.636179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.636984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:23.637422 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:23.638985 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:23.639832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:23.640894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:23.641825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 107/226] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:23.648825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.655211 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:23.665007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 120/227] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:23.666112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.764351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:23.836917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8469a9-da53-4bfd-a71d-e06f19a6959f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.849170 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b971a36-76fc-4a0b-8cf7-26ef8b3d61ca None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:23.899631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.900858 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.901577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.902527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.903272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:23.904125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:23.904889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:23.905336 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:23.906918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:23.907332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:23.907756 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:23.910309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:23.913030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:23.917923 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 121/228] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:23.920945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 108/229] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:23.926441 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:23.927779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:24.036293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:24.099306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477928f7-b2ca-4d7c-a9a8-fc7c50df5309 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:24.122838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d811f-bd74-488b-94c6-4a5756f96bf9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:24.147311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:24.159938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:24.160546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:24.160824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:24.161093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:24.162342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:24.163819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:24.165440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 109/230] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:24.169352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:24.170011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:24.170282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:24.170551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:24.171672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:24.173119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:24.174907 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 122/231] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:23 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:24.182383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:24.192974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:24.336068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f17a2ab-9e60-47b0-849c-a4dbd2405321 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:24.357934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222862b4-e23b-4d95-829c-b270a7f1afc1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:24.374936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:24.375981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:24.376252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:24.376524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:24.377783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:24.379478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:24.381757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 123/232] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:24 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:24.388041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:24.394264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:24.426402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:24.427000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:24.427275 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:24.427544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/300ad085fe6143ad9ab831560faeacf8/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:24.431502 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, user_id=300ad085fe6143ad9ab831560faeacf8, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:24.432980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fcdd8-7453-4093-96b5-dd64afce9c67 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:24.462928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 110/233] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:24 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/300ad085fe6143ad9ab831560faeacf8/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:24.546357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c738f6b-c3b8-4a0c-916c-3c55476243a1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:24.606014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:24.606716 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:24.606990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:24.607258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/300ad085fe6143ad9ab831560faeacf8/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:24.613749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, user_id=300ad085fe6143ad9ab831560faeacf8, role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:24.615286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196a0b93-a55d-4567-b3fe-604e48201118 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:24.640734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 124/234] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:24 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/300ad085fe6143ad9ab831560faeacf8/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:25.211604 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f344-4548-447b-a2eb-816d7270af38 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.212137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f344-4548-447b-a2eb-816d7270af38 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.212137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf7f344-4548-447b-a2eb-816d7270af38 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.213572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 111/235] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:25.219735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d31-3ab2-4434-8322-1a67aaf057d0 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.220256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d31-3ab2-4434-8322-1a67aaf057d0 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.220256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0309d31-3ab2-4434-8322-1a67aaf057d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.249321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b0309d31-3ab2-4434-8322-1a67aaf057d0 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:25.250575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b0309d31-3ab2-4434-8322-1a67aaf057d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:25.322285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:25.330897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 125/236] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:25.343722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00ec5-8e3c-484e-9790-e51f65762cd0 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.344314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00ec5-8e3c-484e-9790-e51f65762cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.344826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee00ec5-8e3c-484e-9790-e51f65762cd0 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.346433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 112/237] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:25.351856 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c71fc-ee97-4f0a-969b-a69166d6a28e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.352141 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c71fc-ee97-4f0a-969b-a69166d6a28e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.352415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c71fc-ee97-4f0a-969b-a69166d6a28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.380181 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-616c71fc-ee97-4f0a-969b-a69166d6a28e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:25.381379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-616c71fc-ee97-4f0a-969b-a69166d6a28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:25.477510 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 126/238] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:25.626143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00ec5-8e3c-484e-9790-e51f65762cd0 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:25.699420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:25.756641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee00ec5-8e3c-484e-9790-e51f65762cd0 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:25.799012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:25.799656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.799924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.800185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.817865 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d7653b4a-32a9-4a22-93a1-3949f0bb85a1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:25.823181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:25.824109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:25.824911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:25.828566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc1a0-0928-4e56-8a04-4472dbba6cb3 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.828852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc1a0-0928-4e56-8a04-4472dbba6cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.829121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dabc1a0-0928-4e56-8a04-4472dbba6cb3 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.829888 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 113/239] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:25.830721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 127/240] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:25.835641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:25.836157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:25.836626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:25.842604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc1a0-0928-4e56-8a04-4472dbba6cb3 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:25.859537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:25.860861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:25.932156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 114/241] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:25.932737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:25.957711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dabc1a0-0928-4e56-8a04-4472dbba6cb3 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:25.986578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.011901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.012514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.012780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.013040 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.013699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:26.014015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.016950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.020232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.039823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 128/242] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:26.045489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.140385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ba5a8-2c2b-4f8a-affb-9681d1270811 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.213313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce16296-d518-4677-b48d-962d58c75777 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.214167 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.214809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.215080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.215347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.216490 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.224099 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.225730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 115/243] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:26.231574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.254437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.255111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.255390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.255705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] PATH_INFO: `/v3/users/eb6e9b4615074e24819cf26129e42e52` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.281249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] RBAC: Authorizing `identity:get_user(user_id=eb6e9b4615074e24819cf26129e42e52)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.282766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.284793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 129/244] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/users/eb6e9b4615074e24819cf26129e42e52 => generated 297 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:26.285717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.339152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.376794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ee6ed-2729-4e5e-a9cf-2448a09258b4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.388818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.400214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf92cfb-7cb6-4d85-9083-aa03eabfa6bd None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.414318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.415426 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.415761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.416100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.416825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.418623 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.443639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.444281 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.444553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.444825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.445527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 116/245] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:26 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:26.446651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.459959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:26.466092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:26.467094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:26.468253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 130/246] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:26.475747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.584047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952e13d-b3a7-4fb8-8bbe-845381f98e6b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.605816 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98699b16-88e5-432c-bbbc-4c459c3431f3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.611362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.641152 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.642118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.642916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.643333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.695323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.697064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.699180 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.699606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.700434 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:26.702060 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.703651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.715874 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:26.723143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 131/247] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:26.735255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.760479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.762205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.872119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1f78c4-df2f-4013-b547-6690eef35786 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:26.908228 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 117/248] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:26.915194 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.922292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:26.922946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:26.923231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:26.923501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] PATH_INFO: `/v3/users/42e846222023460ea0ec229b9f697d85` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:26.924651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] RBAC: Authorizing `identity:get_user(user_id=42e846222023460ea0ec229b9f697d85)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:26.927192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:26.929580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 132/249] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/users/42e846222023460ea0ec229b9f697d85 => generated 297 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:26.935208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:26.984246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:27.014608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:27.058051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4902f4d-0ece-46a6-8e1f-6c317e33e55d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.075971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ddde97-798a-4949-b574-5a71a0952602 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.109628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.110295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.110615 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.110823 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.114097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.115743 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84beaf0-881f-44f6-b3ba-a9c5ffc9b9d4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.117647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 118/250] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:27.119063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e813-0935-4e1d-83f6-6f88ee96f8d8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.119178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e813-0935-4e1d-83f6-6f88ee96f8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.119394 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d617e813-0935-4e1d-83f6-6f88ee96f8d8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.120857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 119/251] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:27.123057 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.123956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.124428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.124891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.125700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e813-0935-4e1d-83f6-6f88ee96f8d8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:27.126595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.128249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d664399-0dff-4ff5-968f-484f5b31abab None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.131653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 133/252] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:26 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:27.132830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.133107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.133380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.159598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:27.161484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:27.239569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d617e813-0935-4e1d-83f6-6f88ee96f8d8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.243209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 134/253] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:27.244248 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:27.278546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:27.300285 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.301040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.301314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.301581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.302450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=eb6e9b4615074e24819cf26129e42e52) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:27.302755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.304262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.307356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:27.319556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 120/254] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:27 2022] GET /identity/v3/role_assignments?user.id=eb6e9b4615074e24819cf26129e42e52&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:27.376123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986b9fcc-cf28-4b15-8ef1-2df25f3422cc None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.432311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.433384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.433872 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.434363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/42e846222023460ea0ec229b9f697d85/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.436435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, user_id=42e846222023460ea0ec229b9f697d85, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.438116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189d4dd-5901-42d3-ab00-a74ae394fb74 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.465972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 135/255] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:27 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/42e846222023460ea0ec229b9f697d85/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:27.570951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:27.685633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45031c9-6fb8-454d-86c4-eb551bb852bd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.688873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b592-b804-4622-8c66-560520c62b05 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.689121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b592-b804-4622-8c66-560520c62b05 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.689386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f04b592-b804-4622-8c66-560520c62b05 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.691281 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 136/256] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:27.697836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.698731 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.699490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.723013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:27.724524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.725059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:27.725164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.725419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.725683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.737561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:27.743424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:27.744369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:27.745154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 121/257] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:27.751224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:27.841077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:27.873977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 137/258] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:27.913769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c45abe-875b-4d68-b1c4-8b35357d36c1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:27.950412 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:27.951390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:27.951859 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:27.952345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:27.953213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:27.953702 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:27.955658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:27.965124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 122/259] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:27.973067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.015800 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:28.117335 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2052e4d5-2acc-41f7-ae63-21e76efb2552 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:28.157674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:28.158340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.158613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.158879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.160002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:28.161513 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:28.163135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 138/260] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:27 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:28.168715 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.291855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c6faa4-6811-48a3-9505-ac69e015655f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:28.303468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.326096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:28.353431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:28.354070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.354351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.354619 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.439712 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:28.464107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdf555-3226-4501-a21f-7e006189f8be None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:28.486989 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:28.488534 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:28.554708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:28.555356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.555631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.555900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.571098 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:28.584321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:28.585251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:28.586071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 139/261] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:28.598004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.627683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 123/262] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:28.633645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.670357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:28.779744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f86a295-4bf6-4346-897b-0a1065d9b68c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:28.825866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:28.826545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.826811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.827073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.829707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:28.831237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:28.833321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 124/263] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:28.839198 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.880712 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0250e32-9917-4c12-b848-ffbcd8da34e6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:28.946699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:28.947327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:28.947613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:28.947887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:28.948561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:28.948868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:28.954009 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:28.962919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 140/264] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:28.969388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:28.992300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.002151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.032204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957a57b5-f738-47f5-b93e-bbffc90c29c8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:29.107519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:29.108024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.108302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.108570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.109300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=300ad085fe6143ad9ab831560faeacf8) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:29.109603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:29.111082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:29.119781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 125/265] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/role_assignments?user.id=300ad085fe6143ad9ab831560faeacf8&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:29.138288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cb3ae-ebaf-4036-a7a9-44f84ca76a16 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:29.193197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:29.193863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.194230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.194432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] PATH_INFO: `/v3/users/300ad085fe6143ad9ab831560faeacf8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.197148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] RBAC: Authorizing `identity:get_user(user_id=300ad085fe6143ad9ab831560faeacf8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:29.198744 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:29.200382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 141/266] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:28 2022] GET /identity/v3/users/300ad085fe6143ad9ab831560faeacf8 => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:29.205977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:29.338144 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b988825-116d-4cbb-9ef9-629891208b95 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:29.384379 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:29.385002 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.385276 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.385542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.412561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.477832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:29.479356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:29.608235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 126/267] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:29.613811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:29.708421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.734693 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e29916-82a1-4e49-a48b-06feb2a8b4f4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:29.775853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:29.776534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.776813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.777082 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.782726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:29.784202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d76cf-c12d-4bc9-9c25-73a4cf68bcf8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:29.786109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 142/268] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:29 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:29.791791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:29.847992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71e82-0170-47b8-8549-be3c473fbc00 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.848318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71e82-0170-47b8-8549-be3c473fbc00 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.848520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d71e82-0170-47b8-8549-be3c473fbc00 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.850056 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 143/269] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:29.852670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:29.856371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:29.856650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:29.856933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:29.880342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:29.881539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:29.945644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cf5f6-734c-4999-bea0-9aea130cee9c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.005733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.006538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.007014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.007450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.008415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=300ad085fe6143ad9ab831560faeacf8) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:30.008576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.010013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7079bb0-988a-46e7-b9fd-7cdc0142cf6a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.015969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 144/270] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:30.024516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 127/271] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:29 2022] GET /identity/v3/role_assignments?user.id=300ad085fe6143ad9ab831560faeacf8&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:30.387305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.477533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:30.514258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0e1f-d6fd-46f7-9ab8-42b02b211268 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.514258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0e1f-d6fd-46f7-9ab8-42b02b211268 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.514258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cf0e1f-d6fd-46f7-9ab8-42b02b211268 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.515232 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e5de8b-57b8-4718-91a3-d1a3bb4e902c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.515590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 128/272] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:30.522493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a08b-6b87-4ab1-822d-f72f2d010263 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.523107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a08b-6b87-4ab1-822d-f72f2d010263 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.524061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c49a08b-6b87-4ab1-822d-f72f2d010263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.546604 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-2c49a08b-6b87-4ab1-822d-f72f2d010263 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:30.548409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-2c49a08b-6b87-4ab1-822d-f72f2d010263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.566150 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.566831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.567107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.567397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.579838 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:30.585270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:30.586184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:30.586978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:30.588704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 145/273] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:30.593843 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.632325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 129/274] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:30.633737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f087-131e-45a6-9f14-f85620b70bd5 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.634055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f087-131e-45a6-9f14-f85620b70bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.634312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077f087-131e-45a6-9f14-f85620b70bd5 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.640304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 130/275] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:30.640304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135772-20ba-423b-b039-9b6a30d4c057 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.640304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135772-20ba-423b-b039-9b6a30d4c057 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.640304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9135772-20ba-423b-b039-9b6a30d4c057 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.640304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 131/276] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:30.643317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b16fe0-449d-455c-915a-740546447726 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.643602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b16fe0-449d-455c-915a-740546447726 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.643946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b16fe0-449d-455c-915a-740546447726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.662150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-d2b16fe0-449d-455c-915a-740546447726 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:30.663348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-d2b16fe0-449d-455c-915a-740546447726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.703459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3456a-02b9-46a2-ac6a-c264ac386eec None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.704511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:30.737946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 132/277] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:30.737946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.737946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.737946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.749493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:30.750720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:30.751736 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.752414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.752660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.752972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.753639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:30.754625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.755448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.760858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:30.765075 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 146/278] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:30.770795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.828860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 133/279] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:30.872340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f415b-266b-4a4d-854b-1ecd3fcbb512 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:30.899007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.904802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:30.905437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:30.905711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:30.905977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] PATH_INFO: `/v3/users/b432b6d1832146e0a7d7f392f79dd495` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:30.919915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] RBAC: Authorizing `identity:get_user(user_id=b432b6d1832146e0a7d7f392f79dd495)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:30.921385 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:30.923173 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 147/280] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/users/b432b6d1832146e0a7d7f392f79dd495 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:30.928932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:30.988603 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.004923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3eb13a6-79a2-4075-b5d3-84722801a00c None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.012737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.037286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7548d-e98d-4340-91e8-f1931d2e2247 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.047565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.048644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.049466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.050097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.062708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:31.070262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:31.071399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:31.072166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 134/281] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:31.074999 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.081681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.082343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.082617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.082881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.083606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, user.id=b432b6d1832146e0a7d7f392f79dd495, scope.system=all) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:31.083908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.085339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.091282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 148/282] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 14 01:17:30 2022] GET /identity/v3/role_assignments?user.id=b432b6d1832146e0a7d7f392f79dd495&scope.system=all&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:31.091892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.204693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c342f61d-f8b4-4e22-8f3a-db12613586db None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.204889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5af498-b0c5-4ebf-b994-ce1bb4afeb12 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.230163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.242627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.243299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.243530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.243808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.244467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.245034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.245307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.245586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.246287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:31.246585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.247980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.255690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:31.257283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 149/283] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:31.260795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:31.261698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:31.262929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 135/284] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:31.264119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.267121 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.295769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.374107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e66fc7-8e90-42d5-9627-942fc9114182 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.374312 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d59c1-b84e-49b7-982d-f0db37c239a4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.414243 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.414885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.415158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.415424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.416586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.417202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.417838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.419586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.419691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.419691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] PATH_INFO: `/v3/users/eb6e9b4615074e24819cf26129e42e52` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.419832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 150/285] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.425575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] RBAC: Authorizing `identity:get_user(user_id=eb6e9b4615074e24819cf26129e42e52)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.426514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.433264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 136/286] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/users/eb6e9b4615074e24819cf26129e42e52 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.433264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.481122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.531431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.531966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdd8630-c22c-499a-8ed9-8e841237d076 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.540018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1841882a-e85b-4722-b7a6-8bfd52348817 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.570191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.570811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.571196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.571467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.572130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:31.572426 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.573798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.576377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.576993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.577267 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.577535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.578221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.580089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.582428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 151/287] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.583240 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.607065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 137/288] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:31 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:31.619118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.715283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2897179d-4792-494a-9ef8-b7f853c43b7a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.734037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.755359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.756519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.756519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.756915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.758439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.760068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.766084 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 152/289] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.769849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.791130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d6ed8-06a5-4676-a645-305528fc1091 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.829777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.830700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.831191 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.831642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] PATH_INFO: `/v3/users/42e846222023460ea0ec229b9f697d85` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.832988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] RBAC: Authorizing `identity:get_user(user_id=42e846222023460ea0ec229b9f697d85)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.834620 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.841035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 138/290] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/users/42e846222023460ea0ec229b9f697d85 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:31.842476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:31.867482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:31.899571 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44398b60-e899-4bee-b6ed-b9bb2ee75527 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:31.940476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:31.941150 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:31.941478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:31.941781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/eb6e9b4615074e24819cf26129e42e52/roles/e863c8d690274edbabdf68d32b2c88fe` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:31.943994 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, user_id=eb6e9b4615074e24819cf26129e42e52, role_id=e863c8d690274edbabdf68d32b2c88fe)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:31.945459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:31.963239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 153/291] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:17:31 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/users/eb6e9b4615074e24819cf26129e42e52/roles/e863c8d690274edbabdf68d32b2c88fe => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:31.973872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454a587f-474a-4adf-b9c8-1e93b00144e2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.017893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.018475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.018756 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.019028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.105740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.107269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.129263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:32.216850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 139/292] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:17:32.222861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.270450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:32.328384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ed1b1-948d-429c-bf23-fbb3de2f01d7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.362968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.363895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.364233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.364559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.365746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.367287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0621a-c1b9-4c52-b56d-5928bff84a40 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.369223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 154/293] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:32.374991 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:32.506754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d1ab04-98c2-48ed-b4f5-8945e1152587 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:32.533026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:32.560064 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:32.560683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:32.560953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:32.561218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:32.561954 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=42e846222023460ea0ec229b9f697d85) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:32.562281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:32.563759 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb0529a-ee96-4920-b1f9-c610de82ef54 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:32.574787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 140/294] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:32 2022] GET /identity/v3/role_assignments?user.id=42e846222023460ea0ec229b9f697d85&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:33.399043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99430-856c-4df5-b892-b9475b710528 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:33.399930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99430-856c-4df5-b892-b9475b710528 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:33.400448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b99430-856c-4df5-b892-b9475b710528 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:33.402010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 155/295] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:33.407469 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:33.407959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:33.408435 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:33.428919 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:33.430484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:33.500412 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 141/296] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:33.766767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b99430-856c-4df5-b892-b9475b710528 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:33.870023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b99430-856c-4df5-b892-b9475b710528 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:33.881063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:33.905790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:33.906460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:33.906735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:33.907006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:33.918096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:33.922955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:33.923813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:33.924578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 156/297] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:33.930461 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.029960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9706649-343c-4cad-a2aa-1fce835573bb None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.044123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:34.069543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.070192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.070467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.070732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.071408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:34.071705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:34.073104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:34.086384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 142/298] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:34.088129 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.200004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75d8dc-f520-4b4b-bf66-660960be1ce7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.235637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.236307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.236579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.236846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] PATH_INFO: `/v3/users/b432b6d1832146e0a7d7f392f79dd495` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.238100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] RBAC: Authorizing `identity:get_user(user_id=b432b6d1832146e0a7d7f392f79dd495)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:34.239754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e7e08-d422-444a-8334-a6dfd0ffb785 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:34.241557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 157/299] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:34 2022] GET /identity/v3/users/b432b6d1832146e0a7d7f392f79dd495 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:34.247564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.259218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a498-13fc-4fef-9a30-2ea123191e11 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.259574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a498-13fc-4fef-9a30-2ea123191e11 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.259755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b6a498-13fc-4fef-9a30-2ea123191e11 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.261556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 158/300] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:34.267063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955099b4-9b82-4b57-a642-9372ee377882 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.267341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955099b4-9b82-4b57-a642-9372ee377882 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.267609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955099b4-9b82-4b57-a642-9372ee377882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.286684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-955099b4-9b82-4b57-a642-9372ee377882 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:34.287809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-955099b4-9b82-4b57-a642-9372ee377882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:34.320589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:34.365202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 159/301] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:34.366337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f627b-670d-4e19-afcc-3b4eea4b068d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.366617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f627b-670d-4e19-afcc-3b4eea4b068d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.366884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f627b-670d-4e19-afcc-3b4eea4b068d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.368488 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 160/302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:34.370920 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec5c4d6-88b0-4980-8c9a-8465eba2c0cd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.374120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ba61-8c8f-44d5-99a5-3c1c5bc609db None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.374405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ba61-8c8f-44d5-99a5-3c1c5bc609db None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.374668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d5ba61-8c8f-44d5-99a5-3c1c5bc609db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.394938 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-49d5ba61-8c8f-44d5-99a5-3c1c5bc609db None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:34.396227 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-49d5ba61-8c8f-44d5-99a5-3c1c5bc609db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:34.413484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.414571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.415118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.415675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] PATH_INFO: `/v3/system/users/b432b6d1832146e0a7d7f392f79dd495/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.417463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b432b6d1832146e0a7d7f392f79dd495, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:34.419247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412aa60-efae-4bf2-9220-a4b03588cedc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:34.432440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 143/303] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Sep 14 01:17:34 2022] PUT /identity/v3/system/users/b432b6d1832146e0a7d7f392f79dd495/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:34.481364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 161/304] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:34.509761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500103b-9e92-43b0-92c7-abbc94ca1008 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.510082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500103b-9e92-43b0-92c7-abbc94ca1008 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.510310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2500103b-9e92-43b0-92c7-abbc94ca1008 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.511844 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 144/305] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:34.517684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.518310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.518807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.538818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:34.540366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:34.621376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 162/306] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:34.631786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500103b-9e92-43b0-92c7-abbc94ca1008 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.676673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:34.743458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2500103b-9e92-43b0-92c7-abbc94ca1008 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.756095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.786261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:34.787442 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.788248 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.788568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.788879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.801527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:34.808430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:34.809316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:34.810269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 145/307] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:34.816714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.875552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da36e2-325f-4f06-94fa-2fe8938c169e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.911077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.911718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.911990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.912256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.913529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:34.915124 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:34.921529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 163/308] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:34.921529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:34.937170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:34.943132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b7895-7418-45ef-8981-de8c4ea7c71f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:34.977790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:34.978460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:34.978690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:34.978957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:34.979628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:34.979947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:34.981342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:34.995940 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 146/309] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:34.996680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.024267 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.030194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c2b77b-4ec6-4e18-be47-42649cb9e661 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.067524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.068614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.068896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.069472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.084574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:35.089929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:35.090812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:35.091610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 164/310] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:35.092284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.113089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf778a7-7927-42e9-9cfc-e0552d965dc2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.150719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.151335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.151610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.151878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] PATH_INFO: `/v3/groups` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.152506 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.153978 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.182026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 147/311] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Sep 14 01:17:34 2022] POST /identity/v3/groups => generated 229 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:35.182026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.202503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed018ccf-0811-492e-8bf2-323a9a362a49 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.214408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.248090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.248746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.249020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.249286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] PATH_INFO: `/v3/users/9feb42c94ec04179833fd3193af0f98f` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.265606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] RBAC: Authorizing `identity:get_user(user_id=9feb42c94ec04179833fd3193af0f98f)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.267282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.269374 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 165/312] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/users/9feb42c94ec04179833fd3193af0f98f => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:35.275533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.295109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.322580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81e3fb-618d-4df6-9364-e884c24a083c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.379930 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.380574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.380913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.381687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.382096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:35.382407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.384422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.394030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 148/313] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:35.401280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.420394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712224b-acd1-47e8-b1ac-79df23047bf7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.466849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.467550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.467830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.468106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.468821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, user.id=9feb42c94ec04179833fd3193af0f98f, scope.system=all) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:35.469122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.470782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.476862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 166/314] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/role_assignments?user.id=9feb42c94ec04179833fd3193af0f98f&scope.system=all&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:35.505070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.526516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc97d0f-6740-462b-aa86-426f7f05660a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.564562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.565562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.565840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.566142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] PATH_INFO: `/v3/users/eb6e9b4615074e24819cf26129e42e52` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.567386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] RBAC: Authorizing `identity:get_user(user_id=eb6e9b4615074e24819cf26129e42e52)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.568882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.570713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 149/315] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/users/eb6e9b4615074e24819cf26129e42e52 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:35.579251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:35.672744 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.695527 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f572b-3e73-4468-9549-4bb5bd65e461 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:35.731676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:35.732294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:35.732566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:35.732829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:35.799793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:35.801305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:35.882388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:35.904044 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 167/316] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:35.909701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:36.010244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df345f4-d476-4660-9c1c-a6650477f222 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:36.049809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:36.050475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:36.050816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:36.051092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:36.052243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:36.053720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7990f198-5a88-4025-bacb-c8facb4c9ae8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:36.056394 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 150/317] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:35 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:36.062520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:36.173787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a84cbf-55e3-46c0-aca5-d37d081c8b5d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:36.209413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:36.210469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:36.210975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:36.211444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:36.212405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e, user.id=eb6e9b4615074e24819cf26129e42e52) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:36.212928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:36.214689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c335a09-83db-4a87-ad57-d67dce2b6de2 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:36.226113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 168/318] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:17:36 2022] GET /identity/v3/role_assignments?user.id=eb6e9b4615074e24819cf26129e42e52&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:36.653309 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09252-7600-456d-a52b-9b3cdbf5c5bc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:36.653712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09252-7600-456d-a52b-9b3cdbf5c5bc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:36.653808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b09252-7600-456d-a52b-9b3cdbf5c5bc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:36.655404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 151/319] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:36.660492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:36.660942 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:36.661014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:36.679268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:36.680429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:36.717302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:36.750114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 169/320] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:37.007367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b09252-7600-456d-a52b-9b3cdbf5c5bc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:37.028272 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:37.124399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b09252-7600-456d-a52b-9b3cdbf5c5bc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:37.158555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:37.159193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.159458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.159717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.169884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:37.175153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:37.176074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:37.176892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:37.181244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 152/321] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:37.182121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:37.280030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedba708-3d84-40eb-8c87-470646a46884 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:37.312987 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:37.313594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.313863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.314163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.314817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:37.315473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:37.316871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:37.325171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 170/322] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:37.330758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:37.412817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:37.437617 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ced0d25-35b9-4c1c-b724-921283468ce5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:37.472640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:37.473278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.473545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.473809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] PATH_INFO: `/v3/users/b432b6d1832146e0a7d7f392f79dd495` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.474962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] RBAC: Authorizing `identity:get_user(user_id=b432b6d1832146e0a7d7f392f79dd495)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:37.476419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0478adb6-72fb-419b-aa29-57eda5cfab16 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:37.478204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 153/323] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:37 2022] GET /identity/v3/users/b432b6d1832146e0a7d7f392f79dd495 => generated 293 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:37.484057 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:37.514740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130d782-e57e-40c6-bfaf-f4e42b83ea8d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.515052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130d782-e57e-40c6-bfaf-f4e42b83ea8d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.515280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a130d782-e57e-40c6-bfaf-f4e42b83ea8d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.516644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 154/324] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:37.520149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:37.521763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113453-1082-409c-8cac-64e34f4de725 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.522067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113453-1082-409c-8cac-64e34f4de725 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.522344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca113453-1082-409c-8cac-64e34f4de725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.542703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-ca113453-1082-409c-8cac-64e34f4de725 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:37.543857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ca113453-1082-409c-8cac-64e34f4de725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:37.599876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cb4f5-7cc2-4c96-bca4-154b4a8038cc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:37.616166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 155/325] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:37.635074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:37.635676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.635948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.636216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.636920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] RBAC: Adding query filter params (role.id=6ffb3f0ff11547d7b4483fc06279ec15, user.id=b432b6d1832146e0a7d7f392f79dd495, scope.system=all) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:37.637217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:37.638682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbf533-3aa9-4c2f-9b18-06d6e38a5e72 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:37.650517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 171/326] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 14 01:17:37 2022] GET /identity/v3/role_assignments?user.id=b432b6d1832146e0a7d7f392f79dd495&scope.system=all&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:37.650688 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6d6a6-e810-47f0-9dd4-63cb9fa600dc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.650768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6d6a6-e810-47f0-9dd4-63cb9fa600dc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.651027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6d6a6-e810-47f0-9dd4-63cb9fa600dc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.652378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 156/327] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:37.658030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:37.658330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:37.658581 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:37.679818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:37.681053 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:37.758961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 172/328] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:37.881893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6d6a6-e810-47f0-9dd4-63cb9fa600dc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:37.960508 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:37.984592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6d6a6-e810-47f0-9dd4-63cb9fa600dc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.018470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.019091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.019365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.019738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.020967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:38.021683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:38.022433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:38.024215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 157/329] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:38.029542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:38.047866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:38.128143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bead8-ec52-4c6b-a9b0-10dc5ab041fb None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.130432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e6937f-3bd8-4ef0-848f-bc7f6d8b7b4d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.160918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.161476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.161745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.162015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.163384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.163988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.164258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.164521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] PATH_INFO: `/v3/groups` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.165145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:38.166626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:38.172218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:38.177083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:38.178066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:38.179051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 173/330] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:38.184029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:38.185328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 158/331] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Sep 14 01:17:38 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:38.273384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:38.293019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295ae5c-eae5-43e0-9178-5bc9a724f9e8 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.329658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.330331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.330604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.330869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.331545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:38.331846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:38.333309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:38.342501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 174/332] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:38.348386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:38.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:38.453631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac65a-269f-4715-98b9-260e0dc52cf3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.489585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.490253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.490525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.490790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] PATH_INFO: `/v3/users/9feb42c94ec04179833fd3193af0f98f` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.491924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] RBAC: Authorizing `identity:get_user(user_id=9feb42c94ec04179833fd3193af0f98f)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:38.493383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe9da0f-d0ec-40e8-b7ec-5e178e91a269 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:38.495040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 159/333] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:38 2022] GET /identity/v3/users/9feb42c94ec04179833fd3193af0f98f => generated 293 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:38.501133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:38.604311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee474419-f08f-4801-9a5f-b96ec8730487 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:38.641090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:38.642188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:38.642668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:38.643134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] PATH_INFO: `/v3/system/users/9feb42c94ec04179833fd3193af0f98f/roles/e863c8d690274edbabdf68d32b2c88fe` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:38.644831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9feb42c94ec04179833fd3193af0f98f, role_id=e863c8d690274edbabdf68d32b2c88fe)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:38.646566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d054c10-f624-4ab6-a6bb-3cd59afbfe7c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:38.666057 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 175/334] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Sep 14 01:17:38 2022] PUT /identity/v3/system/users/9feb42c94ec04179833fd3193af0f98f/roles/e863c8d690274edbabdf68d32b2c88fe => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:40.860003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3bd-516c-42a0-9584-1926bac161d2 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:40.860929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3bd-516c-42a0-9584-1926bac161d2 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:40.861372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528d3bd-516c-42a0-9584-1926bac161d2 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:40.863255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 160/335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:40.871703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc658-2377-46d4-91c2-d99da515cd92 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:40.872548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc658-2377-46d4-91c2-d99da515cd92 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:40.872989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcfc658-2377-46d4-91c2-d99da515cd92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:40.902224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-fbcfc658-2377-46d4-91c2-d99da515cd92 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:40.903739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-fbcfc658-2377-46d4-91c2-d99da515cd92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:40.906178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:40.990689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 176/336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:41.125371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24cc0-477a-4fc8-8696-318d88193eea None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.126014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24cc0-477a-4fc8-8696-318d88193eea None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.126486 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b24cc0-477a-4fc8-8696-318d88193eea None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.128101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 161/337] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:41.133663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c7-6049-49d7-bdb9-e1d5c20f9260 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.133941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c7-6049-49d7-bdb9-e1d5c20f9260 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.134239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007954c7-6049-49d7-bdb9-e1d5c20f9260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.158991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-007954c7-6049-49d7-bdb9-e1d5c20f9260 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:41.160210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-007954c7-6049-49d7-bdb9-e1d5c20f9260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:41.248067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 177/338] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:41.353503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d67e4-31b7-4fde-bc35-43984c391517 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.353503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d67e4-31b7-4fde-bc35-43984c391517 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.353503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71d67e4-31b7-4fde-bc35-43984c391517 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.354280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 162/339] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:41.358637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.358917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.359185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.383096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:41.384215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:41.456290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:41.461907 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 178/340] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:41.535910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d67e4-31b7-4fde-bc35-43984c391517 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:41.618512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:41.659445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71d67e4-31b7-4fde-bc35-43984c391517 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:41.700767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:41.701398 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.701671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.701933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.718981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:41.724004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:41.724889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:41.726344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 163/341] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:41.731803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:41.731939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:41.867673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1904b9-4cc1-4d2f-8fbc-1eb3202dd629 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:41.902371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:41.905800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:41.906710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.907129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.907476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.908393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:41.908817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:41.909998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3535e-8ef6-4b56-aef3-cc9c88e617d1 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:41.910738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f6c43-d7e7-4b72-85ff-3a01304222f2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:41.921020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 164/342] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:41.923177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650446-d624-4245-8fb4-f6fe6dd593bc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.923177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650446-d624-4245-8fb4-f6fe6dd593bc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.923177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16650446-d624-4245-8fb4-f6fe6dd593bc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:41.924377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 165/343] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:41.929352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16650446-d624-4245-8fb4-f6fe6dd593bc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:41.974687 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:41.975729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:41.976249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:41.976731 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.000154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b926cc39-fdb6-4de0-a67f-f8c17a700747 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:42.009029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:42.009915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:42.010741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:42.014147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 179/344] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:42.016145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.016631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.017106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.042467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16650446-d624-4245-8fb4-f6fe6dd593bc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.048812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:42.050473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:42.088472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23050fee-5824-4213-8a56-f825009b840b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.089111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050fee-5824-4213-8a56-f825009b840b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.089381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050fee-5824-4213-8a56-f825009b840b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.089644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23050fee-5824-4213-8a56-f825009b840b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.154944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.177517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 180/345] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:42.178243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.178351 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.188439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23050fee-5824-4213-8a56-f825009b840b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:42.190332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23050fee-5824-4213-8a56-f825009b840b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:42.308552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ba26ea-8450-488d-b4b5-65463477ba73 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.309541 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 166/346] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:42.310603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23050fee-5824-4213-8a56-f825009b840b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.354732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.355517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.355796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.356065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.356737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:42.357039 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:42.358663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b903beae-2f28-404a-a3b0-2953d0b4de29 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:42.368770 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 181/347] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:42.370189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c92-a59a-4624-afd4-5e3a594666ad None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.370470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c92-a59a-4624-afd4-5e3a594666ad None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.370737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47177c92-a59a-4624-afd4-5e3a594666ad None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.372105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 182/348] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:42.372967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c92-a59a-4624-afd4-5e3a594666ad None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.419721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.432529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23050fee-5824-4213-8a56-f825009b840b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.443271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.475134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.475824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.476048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.476313 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.487932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:42.493013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:42.494080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:42.494900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:42.495521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 167/349] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:42.496093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.505146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47177c92-a59a-4624-afd4-5e3a594666ad None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.558778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.559599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.559879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.560148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.561573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:42.563069 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e9a9c3-5a9a-473b-9c10-5b042d58ef3b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:42.568547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 183/350] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:42.568547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.568547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.568547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.591801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:42.593088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:42.624558 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8307aa93-05dd-40bb-a76e-bbf4fbd8e689 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.665031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.665740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.666145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.666485 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.689335 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 184/351] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:42.690835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.694548 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.747654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:42.749972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:42.751549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:42.810952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc64feb8-732e-43d8-9f79-32d471673c7e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:42.858740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:42.859773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:42.860341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:42.860798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:42.872665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:42.878398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:42.879353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 14 01:17:42.880134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:42.880717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 168/352] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:42.881108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.882898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 185/353] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:42.884037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:42.954463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.006048 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca35b7-e956-429c-b4ff-37378698fc00 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.011212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4f1b9-51b8-439c-b99f-78227a1bdde4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.017473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.049513 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.049931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.050223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.050499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.050633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.050793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.050898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.051158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.051830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:43.052132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.053518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.059486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.061559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.066298 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 169/354] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.066448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 186/355] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:42 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.066560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.066648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.186057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e993f-ff88-452d-ae53-32442c5404bd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.194911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd6705c-4f53-45ab-9cc6-16cecb538206 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.203489 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.226039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.226591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.226866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.227140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.227810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:43.228111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.229513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.236678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.237329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.237602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.237870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.238976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 170/356] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.239160 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.239588 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.241912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.242307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 187/357] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.243079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.290958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.353082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ed323-c25f-4057-bded-5701d72c1dd6 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.363147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610f81d3-fc86-4592-a187-086d6ff3aa2a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.390658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.391425 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.391709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.391967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.404053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:43.404731 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.405363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.405641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.405910 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:43.409770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:43.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:43.411675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:43.415601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 171/358] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:43.415601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.464897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.490158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.492319 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.530252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3401b-d9d9-412e-8179-3a33912cc053 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.538104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.572585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.573270 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.573549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.573808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.574605 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:43.574907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.576841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c8f330-5456-49f6-aad8-f068437d1a3d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.585328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 172/359] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.586952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cea7-c7cf-4e1b-80c1-10455dbf06dd None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.587348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cea7-c7cf-4e1b-80c1-10455dbf06dd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.587732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1cea7-c7cf-4e1b-80c1-10455dbf06dd None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.589854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 173/360] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:43.590850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1cea7-c7cf-4e1b-80c1-10455dbf06dd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.629435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 188/361] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:43.630402 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.731496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1cea7-c7cf-4e1b-80c1-10455dbf06dd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.735337 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.779324 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3fe6cf-785d-44ec-8092-590e893cd73f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.786141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.786802 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.787811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.787811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.789173 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.790732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.792352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 174/362] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.793115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.820930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.821572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.821848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.822151 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] PATH_INFO: `/v3/users/9feb42c94ec04179833fd3193af0f98f` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.823307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] RBAC: Authorizing `identity:get_user(user_id=9feb42c94ec04179833fd3193af0f98f)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.824755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e720e21f-0c68-448a-b647-698e68ff583a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.826386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 189/363] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/users/9feb42c94ec04179833fd3193af0f98f => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.827508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.827783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.828051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.854263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:43.855605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:43.859535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:43.918950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be4d380-420e-472e-acca-d71a6918e353 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:43.945071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 190/364] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:43.945071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:43.963717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:43.964732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:43.965207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:43.965676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:43.966771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:43.967283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:43.968924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:43.982398 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 175/365] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:43.983623 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.022700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.069900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f708cadc-b148-43a0-8a41-d73070ecb57c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.120077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.120728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.121000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.121263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.122462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.123901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.125770 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 191/366] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:44.126412 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.128058 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe34ca-4f99-420b-95ce-0ee963c7d866 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.140858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.168470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.170341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.170814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.171322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.172235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:44.172761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.174991 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.183543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 176/367] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:43 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:44.184631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.294901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11a5ba1-3a76-4174-8019-eec00424dcf9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.336334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.337100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.337383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.338122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.339056 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] RBAC: Adding query filter params (role.id=e863c8d690274edbabdf68d32b2c88fe, user.id=9feb42c94ec04179833fd3193af0f98f, scope.system=all) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:44.339497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.340968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.352844 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.356135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 192/368] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/role_assignments?user.id=9feb42c94ec04179833fd3193af0f98f&scope.system=all&role.id=e863c8d690274edbabdf68d32b2c88fe => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:44.357338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.362773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e46cf-3ba5-45bf-ab0e-192b24feb8fa None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.428041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.430384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.430384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.430384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.510621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.576515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.578111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.598499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afeac17-6b82-4f24-ba6b-8d8581a2c96c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.651529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.652168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.652444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.652812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.653934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.655470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.657440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 193/369] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:44.658066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.720841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.732019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 177/370] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:44.733134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.785464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a4afb-375b-47f9-8f95-af1b9f72aca4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.818349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.823201 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.824142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.824694 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.825335 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.837143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:44.842866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:44.843752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:44.844544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 194/371] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:44.845291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.858050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945c5cdd-1acf-480b-bf04-4b436b456d3a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.894873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.895576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.895847 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.896114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.896846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:44.897136 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.898873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:44.905778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 178/372] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:44.906645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:44.954874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4501e6-d008-4110-bb64-028ac3fe3289 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:44.974751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:44.995121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:44.995754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:44.996025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:44.996292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:44.997664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:44.999336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.001183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 195/373] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:45.007327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:45.030972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f6b643-4d83-4cfd-845a-60c5b47fa1bc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.092130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.092778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.093066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.093333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.094061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:45.094367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.095808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.114201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 179/374] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:45.118445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:45.137110 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:45.212570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29b96f1-5365-4aec-a8b5-42fc8f3c3612 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.265191 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c430a8-d25f-459e-963d-8f7cf2822c50 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.282738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.283397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.283681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.283956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.285202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.286732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.288791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 196/375] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:45.290576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:45.294637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:45.304921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.305544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.305820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.306117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.381297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.382728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.508677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137a2868-fe7d-40b7-859e-553f36405e10 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.549040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 180/376] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:45.554720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:45.565826 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:45.580356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.580977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.581251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.581519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.582350 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:45.582658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.584107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.597917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:45.601861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 197/377] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 260 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:45.707428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e829a1e-666d-4087-95ee-1d8d8da75ec0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.753410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.754131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.754409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.754789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] PATH_INFO: `/v3/projects/3f71bf19133e4a47a0d69d2812d255f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.755981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] RBAC: Authorizing `identity:get_project(project_id=3f71bf19133e4a47a0d69d2812d255f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.757476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.759628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 181/378] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/projects/3f71bf19133e4a47a0d69d2812d255f8 => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:45.765381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:45.886917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47093670-6071-436f-8e41-abbc4a264598 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:45.926260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:45.926949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:45.927235 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:45.927508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] PATH_INFO: `/v3/groups/8be3f511fe4846aca48d891b1dc7ee14` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:45.932757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] RBAC: Authorizing `identity:get_group(group_id=8be3f511fe4846aca48d891b1dc7ee14)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:45.934266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2718f0ec-6b32-4ee0-b4c0-9dc638cfabb1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:45.936248 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 198/379] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/groups/8be3f511fe4846aca48d891b1dc7ee14 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:45.942745 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:46.053286 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:46.068805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafafb8a-1c9d-462c-b96d-96c143ad60ac None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:46.118607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:46.119225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:46.119501 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:46.119766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:46.120501 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] RBAC: Adding query filter params (group.id=8be3f511fe4846aca48d891b1dc7ee14, role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=3f71bf19133e4a47a0d69d2812d255f8) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:46.120802 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:46.122330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c2389-da13-460f-9f8b-4a145d6e2a69 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:46.133032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 182/380] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:45 2022] GET /identity/v3/role_assignments?group.id=8be3f511fe4846aca48d891b1dc7ee14&scope.project.id=3f71bf19133e4a47a0d69d2812d255f8&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:46.191363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef90a80-8df5-4ea5-a300-623f42f44015 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:46.191912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef90a80-8df5-4ea5-a300-623f42f44015 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:46.191993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef90a80-8df5-4ea5-a300-623f42f44015 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:46.193466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 199/381] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:46.199410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:46.199901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:46.200344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:46.225501 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:46.227033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:46.321297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 183/382] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:46.630262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef90a80-8df5-4ea5-a300-623f42f44015 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:46.701141 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:46.777819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef90a80-8df5-4ea5-a300-623f42f44015 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:46.820188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:46.820833 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:46.821106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:46.821372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:46.833794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:46.838898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:46.839893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:46.840933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 200/383] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:46.846274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:46.968044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:46.995794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6819532e-23f6-498e-b0b7-7688447bc31e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:47.038326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:47.038947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.039217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.039477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.040159 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:47.040454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:47.041839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc283bc-e352-4099-95bd-570c5c75ad43 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:47.051360 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 184/384] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:47.059555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:47.128355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdcb22-5c4c-4911-9afa-2539fcf20166 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.128955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdcb22-5c4c-4911-9afa-2539fcf20166 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.128955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fdcb22-5c4c-4911-9afa-2539fcf20166 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.130439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 185/385] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:47.135661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.135939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.136200 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.174300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:47.176567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:47.206213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fda16-7f13-4c65-a98a-a9055f53a360 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:47.239570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:47.276907 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:47.277932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.278436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.278900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.280209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:47.281917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:47.284245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 201/386] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:47 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:47.290032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:47.306012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 186/387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:47.422349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aa372-93cb-4d22-ba50-715b02052066 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:47.459780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:47.460825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.461297 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.461752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.463098 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:47.464778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64242fbc-022d-4605-b500-ae25cb47a4c8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:47.466675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 202/388] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:47 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:47.472820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:47.566481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:47.597758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3e95d1-4dfd-4b59-a04b-a23a77bb750a None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:47.637720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:47.638408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.638678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.638944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/groups/0bc72e370bee4958b99ef76197b37921/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.643201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, group_id=0bc72e370bee4958b99ef76197b37921, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:47.644714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:47.672702 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 187/389] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 14 01:17:47 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/groups/0bc72e370bee4958b99ef76197b37921/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:47.795369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61d-4053-4605-923e-ad9ccee5545b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.796091 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61d-4053-4605-923e-ad9ccee5545b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.796576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f450e61d-4053-4605-923e-ad9ccee5545b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.798261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 203/390] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:47.801382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:47.803776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:47.804321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:47.804791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:47.838516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:47.842413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:47.935869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:47.976851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 204/391] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:47.978344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2e7616-8e9a-4b54-82c6-35424e7f4eb4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.040033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.041099 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.041568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.042050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.045017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:48.054317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:48.059632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:48.060511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:48.061369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 188/392] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:48.069784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.171096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ae49b-8da8-419b-a802-c89fea8e28a5 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.207889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.209307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.209777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.210256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.211120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:48.211612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:48.213203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d98f69a-a154-4207-8d97-70308d0d4df0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:48.221503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 205/393] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:48.227370 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.336932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22d144-e2e2-4c8f-9f8a-b8a571ad6e75 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.372822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1189-d75d-46ab-9684-d8a8356d51b1 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.373517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1189-d75d-46ab-9684-d8a8356d51b1 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.374070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed1189-d75d-46ab-9684-d8a8356d51b1 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.375959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 206/394] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:48.377350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.378240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.378761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.379213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.380528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:48.382220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:48.382452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.382737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.383013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.384877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 189/395] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:48.390410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.412638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:48.413848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:48.475995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:48.477041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:48.513950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 207/396] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:48.515203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.519780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d436-ce0b-4242-8637-c395a37bc87f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.591570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.592538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.592981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.593427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.594748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:48.599240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:48.603621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 190/397] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:48.611624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.703196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528746b1-6acf-4e69-9170-dfebb80661e7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.766801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.767755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.768196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.768643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.781367 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:48.786849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:48.787906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:48.788718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:48.790818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 208/398] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:48.792025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.804260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c02660-b292-453d-b330-7ca8c1e8e2c3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.837473 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:48.838514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:48.843620 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.844418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.845723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.846140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/groups/0bc72e370bee4958b99ef76197b37921/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.848737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] RBAC: Authorizing `identity:create_grant(project_id=e5e8925c1ddd4167825bcccae9db62a5, group_id=0bc72e370bee4958b99ef76197b37921, role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:48.850466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:48.867773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 191/399] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 14 01:17:48 2022] PUT /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5/groups/0bc72e370bee4958b99ef76197b37921/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:48.874033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:48.922095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ad858c-a581-41e8-965d-5655cb68ac9b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:48.966210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:48.966975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.967320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.967637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:48.986244 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-5987ef99-663d-456b-9605-92e5809bcba6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:48.993821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:48.994824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:48.995596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 209/400] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:48.997108 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c400a52-ca0d-42e3-86aa-898fc5cfe60f None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:48.997451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c400a52-ca0d-42e3-86aa-898fc5cfe60f None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:48.998110 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c400a52-ca0d-42e3-86aa-898fc5cfe60f None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.001259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 210/401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:49.002094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c400a52-ca0d-42e3-86aa-898fc5cfe60f None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.002771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248a60b-6ecb-4153-b75d-569b8ca090dd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.041739 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.042763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.043295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.043769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.044714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:49.045239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.046905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2b860f-d46e-4a1b-a95e-9dd4a40ef793 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.057253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 192/402] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:49.059009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.059513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.059971 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.082335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:49.083832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:49.127945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.131710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c400a52-ca0d-42e3-86aa-898fc5cfe60f None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.149070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.172178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.172884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.173227 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.173620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.174047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 193/403] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:49.174387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:49.174750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.175566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.176334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.185721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 211/404] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:49.194132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.355201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6249542-0bb2-4cf4-a165-718760113aba None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.368937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e48f58-4930-4dc4-a6ab-df9a65e5b5c6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.422638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.423288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.423556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.423827 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.424951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.426494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.428109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 194/405] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:49.433806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.438663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.439373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.439992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.440267 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.441448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.442949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.445049 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 212/406] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:49.445806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.483347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.492832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.540269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee91788-0b06-417e-ac9c-4af89854262a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.575203 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.575929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.576203 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.576461 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.577603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.579088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.581401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 195/407] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:49.582422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.587619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d3fedc-2ca5-4cc0-8e20-0610c8063df9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.630786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.632085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.632766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.633187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.646452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:49.653679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:49.654793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:49.655707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:49.658504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 213/408] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:49.659424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.695243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f52ff9c-9d36-4bda-8daa-59d634ddff5b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.717255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.739482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.740510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.740931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.741813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.743341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.745909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.748066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 196/409] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:49.748969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.769317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.791579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21f7df3-6158-49c8-a557-9c02f4f91a95 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.826944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.827985 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.828477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.829053 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/groups/0bc72e370bee4958b99ef76197b37921/roles/6ffb3f0ff11547d7b4483fc06279ec15` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.831462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, group_id=0bc72e370bee4958b99ef76197b37921, role_id=6ffb3f0ff11547d7b4483fc06279ec15)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.833238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.851561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 214/410] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 14 01:17:49 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/groups/0bc72e370bee4958b99ef76197b37921/roles/6ffb3f0ff11547d7b4483fc06279ec15 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:49.852895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.864314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f4fe-4c7c-4311-bdc0-ddd459595711 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:49.904332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:49.905013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:49.905349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:49.905728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:49.906627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:49.906997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:49.908790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:49.923394 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 197/411] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:49.925679 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:49.981129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:49.999652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91869357-27ef-4cfd-a39c-a6a821cc9d25 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.036661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86fade2-8061-4f17-91fe-079a2aea3bd3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.050790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.050790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.053691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.053891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/groups/0bc72e370bee4958b99ef76197b37921/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.061140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] RBAC: Authorizing `identity:create_grant(project_id=b9cfe073c3784988ac77c3a6eb738a6e, group_id=0bc72e370bee4958b99ef76197b37921, role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:50.062708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:50.093711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 215/412] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 14 01:17:49 2022] PUT /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e/groups/0bc72e370bee4958b99ef76197b37921/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:17:50.126977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.127655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.127925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.128192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] PATH_INFO: `/v3/groups/8be3f511fe4846aca48d891b1dc7ee14` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.129657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] RBAC: Authorizing `identity:get_group(group_id=8be3f511fe4846aca48d891b1dc7ee14)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:50.131155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a44880d-f295-47e0-a90a-ad5bbfb40660 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:50.133296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 198/413] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:49 2022] GET /identity/v3/groups/8be3f511fe4846aca48d891b1dc7ee14 => generated 229 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:50.139288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:50.146502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:50.182169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027423-8f92-4acf-bdb8-7704ca8e5ee8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.182772 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027423-8f92-4acf-bdb8-7704ca8e5ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.183255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e027423-8f92-4acf-bdb8-7704ca8e5ee8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.187264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 199/414] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:50.193371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.193371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.193371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.213632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:50.214889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:50.284515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65a2161-01fe-4d8b-908a-751eac4bd914 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.305631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 200/415] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:50.330946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.331743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.331975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.332245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] PATH_INFO: `/v3/projects/3f71bf19133e4a47a0d69d2812d255f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.334284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] RBAC: Authorizing `identity:get_project(project_id=3f71bf19133e4a47a0d69d2812d255f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:50.334930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:50.336958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 216/416] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:50 2022] GET /identity/v3/projects/3f71bf19133e4a47a0d69d2812d255f8 => generated 344 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:50.343107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:50.392793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:50.447076 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fbfcd-17bf-4320-9430-02bfd03c093e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.482304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.482967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.483237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.483507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] PATH_INFO: `/v3/projects/3f71bf19133e4a47a0d69d2812d255f8/groups/8be3f511fe4846aca48d891b1dc7ee14/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.485428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] RBAC: Authorizing `identity:create_grant(project_id=3f71bf19133e4a47a0d69d2812d255f8, group_id=8be3f511fe4846aca48d891b1dc7ee14, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:50.486952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:50.514686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 201/417] 10.222.0.26 () {58 vars in 1442 bytes} [Wed Sep 14 01:17:50 2022] PUT /identity/v3/projects/3f71bf19133e4a47a0d69d2812d255f8/groups/8be3f511fe4846aca48d891b1dc7ee14/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:17:50.656167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:50.745309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:50.777682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e79ee67-2e78-4b48-b02b-22b2779ad9e9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.812015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.812660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.812931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.813199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.824002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:50.829271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:50.830287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:50.831113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 217/418] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:50.836553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:50.945272 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb58bda-625d-42aa-b24d-9e450d98552e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:50.955064 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:50.987710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:50.994267 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:50.994599 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:50.994875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:50.995695 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] RBAC: Adding query filter params (name=member) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:50.995935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:50.997371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bdba5d-ce1d-4621-af8d-023fab53ff96 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:51.006863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 202/419] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:51.012555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:51.172251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e357e5-98ca-4406-82cf-e9c1a917ed35 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:51.203884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:51.210876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:51.211865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:51.212337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:51.212796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:51.288074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:51.289893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:51.386983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5de89-3589-4bfa-a9c4-4b7337e51501 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:51.387303 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5de89-3589-4bfa-a9c4-4b7337e51501 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:51.387872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce5de89-3589-4bfa-a9c4-4b7337e51501 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:51.389934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 203/420] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:51.395474 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:51.396025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:51.396025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:51.403984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 218/421] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:51.409470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:51.443103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:51.443805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:51.444343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:51.550635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfff89a9-9a47-4472-a8ce-0c98d4a3a45e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:51.551213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 204/422] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:51.609038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:51.609944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:51.610375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:51.610776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:51.612249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:51.613962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:51.616083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 219/423] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:51 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:51.621637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:51.747148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:51.750359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926c9e4-099a-40cf-81a0-f2d44e6082f8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:51.791808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:51.792438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:51.792707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:51.792969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:51.794094 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:51.795581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:51.800580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 205/424] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:51 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:51.803924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:51.913966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:51.956248 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:51.960063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be75566-7d4d-4b34-95da-21c3ee0f0ce1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:52.012525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:52.013193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.013468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.013737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.014510 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:52.014997 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:52.016241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:52.029114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 220/425] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:51 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 611 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:52.102891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3aecf55-b51a-41d2-a872-69def8cb5cfe None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:52.153862 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:52.154548 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.154813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.155081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.176355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:52.181331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:52.182707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:52.184057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:52.184825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 206/426] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:52.190530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:52.299103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a944f-7cf0-4ffa-a8e1-c378bd3424de None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:52.334575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:52.335109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.335379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.335645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.336362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:52.336665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:52.338059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:52.350239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 221/427] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:52.352773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:52.382452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:52.459004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7dde42-d2b7-487c-b678-625a8613aee1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:52.507562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:52.508180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.508515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.508781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.581450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:52.583132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5e3e6b-7d02-4873-8fe7-dbf35d6b068d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:52.614418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:52.693278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 207/428] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:52.694751 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:52.707788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed417bf-72fb-48fc-9a4a-5c6d57a93a8b None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.708093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed417bf-72fb-48fc-9a4a-5c6d57a93a8b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.708340 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed417bf-72fb-48fc-9a4a-5c6d57a93a8b None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.713201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 208/429] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:52 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:52.715053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.715327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.715580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.734381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:52.740182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:52.741906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:52.823126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231a7ded-3b16-4230-a7ff-35eb12098cc0 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:52.836149 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 209/430] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:52.878946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:52.879512 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.879787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.880055 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] PATH_INFO: `/v3/projects/e5e8925c1ddd4167825bcccae9db62a5` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.881212 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] RBAC: Authorizing `identity:get_project(project_id=e5e8925c1ddd4167825bcccae9db62a5)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:52.882761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2e2795-9210-4974-888b-c00757f4e454 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:52.884778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 222/431] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:52 2022] GET /identity/v3/projects/e5e8925c1ddd4167825bcccae9db62a5 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:52.890357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:52.955124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:52.968886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed249c-723a-4985-8eaf-3cf6d9f640de None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.969218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed249c-723a-4985-8eaf-3cf6d9f640de None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.969438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed249c-723a-4985-8eaf-3cf6d9f640de None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.971104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 223/432] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:52.976869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:52.977158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:52.977435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:52.995812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:52.997079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:53.005062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc909abd-40f2-4c4a-aa09-59572017ccfb None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.052205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.053115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.053388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.053656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.055007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:53.057068 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:53.060269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 210/433] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:52 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:53.067355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.080776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 224/434] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:17:53.145534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.175859 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.188666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f56805-4e48-401b-aa38-e7905aa889c0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.223582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.240915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.240915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.240915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.241265 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.241830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=e5e8925c1ddd4167825bcccae9db62a5) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:53.242231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:53.244930 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1699698-78fb-496d-bd1a-cbf34d261940 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:53.252734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 211/435] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=e5e8925c1ddd4167825bcccae9db62a5&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 611 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:53.261209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02bd0e-8a6f-45db-b322-9950d3c5dcb4 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.261626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02bd0e-8a6f-45db-b322-9950d3c5dcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.262024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a02bd0e-8a6f-45db-b322-9950d3c5dcb4 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.264249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 212/436] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:53.272249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.272249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.272249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.282583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4b3b30-0e3d-4780-8521-bd68687edb55 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.290545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:53.291917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:53.326358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.326997 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.327274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.327532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.346958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:53.353077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:53.354047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 14 01:17:53.355100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:53.358073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 225/437] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:53.361154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.384274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 213/438] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:53.397438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.458112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.476203 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec360f2-b59c-43d0-b7ef-512212f5d294 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.519504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.520365 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.520813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.521226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.522085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] RBAC: Adding query filter params (name=member) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:53.522526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:53.522649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ef59e8-b29b-4267-b083-9f56e5dfff5f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.525541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:53.537178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 226/439] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:53.543950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.548157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.576283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.576918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.577187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.577448 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.589807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:53.594714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:53.595621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:53.597262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 214/440] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:53.602215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.672370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e0851-a6a6-4677-a570-c672636f7a16 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.714762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.715424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.715701 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.715971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.719714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35dec71-ccc8-4e0e-9c87-65b483eef809 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.733268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.757505 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.758142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.758412 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.758675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.759343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:53.759637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:53.761016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:53.770131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 215/441] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:53.770131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.786965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:53.788418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:53.813221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.868332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8087918-1138-4a94-931e-d181f0ede7e6 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:53.884644 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 227/442] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:17:53.884644 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.903184 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:53.903810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:53.904082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:53.904347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:53.914545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:17:53.919833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:53.920708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:53.921497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 216/443] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:53.922183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:53.937319 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:53.990261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e8cda-5e7e-42a4-8eb3-bbe627ad637d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.017510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5010024e-f816-4532-8d70-7de2e4ecebc0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.019186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.023722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.024332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.024620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.024870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.052747 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.053366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.053636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.053907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.055057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.056515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.062742 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 217/444] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.062742 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.094023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.095534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.140371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.165007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf65442f-7d7c-4fa7-98a1-1fb061033bbd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.194448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 228/445] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:54.195224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.203263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.203869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.204141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.204404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.205067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:54.205369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.206800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.214871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 218/446] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.215655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.248863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.296025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955d6f33-f980-4941-ab8c-c35aa4f4b420 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.318182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6372b3a1-1cd5-4ae5-9fdd-8d9eaff6ce3f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.329890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.330550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.330822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.331095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.332328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.334006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.335585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 229/447] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.336340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.349913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.350602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.350867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.351128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] PATH_INFO: `/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.352271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] RBAC: Authorizing `identity:get_project(project_id=b9cfe073c3784988ac77c3a6eb738a6e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.353685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.354646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.360008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 219/448] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/projects/b9cfe073c3784988ac77c3a6eb738a6e => generated 302 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.360008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.440101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024b4846-e211-46da-848f-11d21c8bd932 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.451740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.454306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d0dfc-f6b8-4bb2-b6fb-a2a01d65d4bd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.472631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.473232 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.473502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.473765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.488214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.488813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.489085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.489352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.490109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=6ffb3f0ff11547d7b4483fc06279ec15, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:54.490445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.491885 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.497510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 220/449] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=6ffb3f0ff11547d7b4483fc06279ec15 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.498301 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.560334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.561836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.578703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.608911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df6b226-3f7e-46f2-a8b6-c933fe06029c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.646347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.647001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.647274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.647545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] PATH_INFO: `/v3/groups/0bc72e370bee4958b99ef76197b37921` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.648853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] RBAC: Authorizing `identity:get_group(group_id=0bc72e370bee4958b99ef76197b37921)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.650368 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.652205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 221/450] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/groups/0bc72e370bee4958b99ef76197b37921 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.658302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.677605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 230/451] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:17:54.683283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.700706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.777557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d5aca5-ece5-4a51-b6b6-e23a7a7cded5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.805414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f90880-8da1-487b-b71f-d3f7a021abb3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:54.817241 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.817869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.818179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.818452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.819190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] RBAC: Adding query filter params (group.id=0bc72e370bee4958b99ef76197b37921, role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:54.819494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.820948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.826104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.827763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 222/452] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/role_assignments?group.id=0bc72e370bee4958b99ef76197b37921&scope.project.id=b9cfe073c3784988ac77c3a6eb738a6e&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.853263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:54.853928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:54.854236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:54.854506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] PATH_INFO: `/v3/projects/3f71bf19133e4a47a0d69d2812d255f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:54.855799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] RBAC: Authorizing `identity:get_project(project_id=3f71bf19133e4a47a0d69d2812d255f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:54.857505 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:54.859521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 231/453] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/projects/3f71bf19133e4a47a0d69d2812d255f8 => generated 344 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:54.865267 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:54.969642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:54.976789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9543af-516d-4714-87d2-7b56923e24f9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:55.012631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:55.013347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:55.013619 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:55.014679 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] PATH_INFO: `/v3/groups/8be3f511fe4846aca48d891b1dc7ee14` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:55.015045 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] RBAC: Authorizing `identity:get_group(group_id=8be3f511fe4846aca48d891b1dc7ee14)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:55.016491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16c9c7-8624-4730-a305-cc602e7d8f26 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:55.018108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 223/454] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Sep 14 01:17:54 2022] GET /identity/v3/groups/8be3f511fe4846aca48d891b1dc7ee14 => generated 229 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:55.024428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:55.128932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554bf56b-aac5-4cc0-bb3d-3c701c667697 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:55.166238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:55.166841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:55.167112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:55.167379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:55.168102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] RBAC: Adding query filter params (group.id=8be3f511fe4846aca48d891b1dc7ee14, role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=3f71bf19133e4a47a0d69d2812d255f8) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:55.168423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:55.169860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175610b-7062-4020-84b3-cb1ee9dd0f97 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:55.176031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 232/455] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Sep 14 01:17:55 2022] GET /identity/v3/role_assignments?group.id=8be3f511fe4846aca48d891b1dc7ee14&scope.project.id=3f71bf19133e4a47a0d69d2812d255f8&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:57.658625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832040f-7e6f-476f-a2fa-05b40a0d7b90 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:57.659155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832040f-7e6f-476f-a2fa-05b40a0d7b90 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:57.659155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832040f-7e6f-476f-a2fa-05b40a0d7b90 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:57.660969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 224/456] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:57.666126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35136b7-d193-452a-b369-58477e2c542a None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:57.666916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35136b7-d193-452a-b369-58477e2c542a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:57.667383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35136b7-d193-452a-b369-58477e2c542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:57.694957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-e35136b7-d193-452a-b369-58477e2c542a None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:57.696485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e35136b7-d193-452a-b369-58477e2c542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:57.755966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:57.792355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 233/457] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:57.839174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a62-440e-40dc-96f6-9174c23b6afa None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:57.839814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a62-440e-40dc-96f6-9174c23b6afa None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:57.840297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6d1a62-440e-40dc-96f6-9174c23b6afa None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:57.842102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 225/458] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:57.847689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:57.847979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:57.848249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:57.869932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:57.871152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:57.989565 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 234/459] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:58.112074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1a62-440e-40dc-96f6-9174c23b6afa None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.247798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6d1a62-440e-40dc-96f6-9174c23b6afa None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.299763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.300481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.300758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.301026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.318415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.319932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:17:58.336446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:58.337423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:17:58.338305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:17:58.338305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:58.338305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:58.342568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 226/460] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:58.348406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.400995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.421201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:58.465412 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4a3f4-a741-40ed-af3b-36879d7d5d76 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.473789 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:58.503262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.503831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.504106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.504366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.505000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:58.505304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.506704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64fe8db-9629-479e-ba19-7fa38d51b00a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.507675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c083b-16d7-4fe0-bccb-f675bcd62464 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.518569 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 235/461] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:58.519654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930cdd-5edd-455d-aad4-cef784c82275 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.519933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930cdd-5edd-455d-aad4-cef784c82275 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.520205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35930cdd-5edd-455d-aad4-cef784c82275 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 236/462] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f71ac-4f1c-47b1-84d5-6cc443c1dd85 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f71ac-4f1c-47b1-84d5-6cc443c1dd85 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f71ac-4f1c-47b1-84d5-6cc443c1dd85 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 237/463] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:17:58.525439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f71ac-4f1c-47b1-84d5-6cc443c1dd85 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.551936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.552868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.553334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.553783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.564882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.566600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-dfddd40e-60f8-4a62-b477-17831cb7700a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:17:58.580793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:58.581842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:17:58.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:17:58.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:58.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:58.584041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 227/464] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:58.585143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c859-ca0a-424a-8159-60de08ebcac6 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.585418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c859-ca0a-424a-8159-60de08ebcac6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.585683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba7c859-ca0a-424a-8159-60de08ebcac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.605665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-aba7c859-ca0a-424a-8159-60de08ebcac6 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:58.606909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-aba7c859-ca0a-424a-8159-60de08ebcac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:58.638955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66f71ac-4f1c-47b1-84d5-6cc443c1dd85 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.666586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:58.672517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 228/465] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:58.674008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.674289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.674552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.685471 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.686717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.687220 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.687808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.688561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.690537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.698068 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:58.699561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:58.718095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 238/466] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:58 2022] POST /identity/v3/users => generated 246 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:58.726219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:58.749205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:58.838590 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f66ad0b-0fce-47fe-b9c6-49ec3f8ba41c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:58.843273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 229/467] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:17:58.876988 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:58.877624 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:58.877896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:58.878196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:58.878993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:58.879345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:58.880744 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:58.895738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 239/468] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:17:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:17:58.901043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.016804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41ab0f-6ded-47dc-bb9e-cba2d2ae9141 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.019460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.047150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.057592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.058653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.059097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.059933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.061007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.063432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.100854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 230/469] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:58 2022] POST /identity/v3/users => generated 248 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:59.139967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.161866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e09f8f3-7cf7-4c0f-b12a-aa1d834a4e19 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.194798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.208083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.208616 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.208886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.209153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.224234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.225817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:17:59.246066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:59.247021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:17:59.247846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:17:59.247846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:59.247846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:17:59.247846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 240/470] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:17:59.253104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.330666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bca0-f14f-4e76-a9b6-c3b04b269498 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.367752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.368652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.368652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.368926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.381025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.382555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.383797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8df824c-f379-47d9-825b-972cbb5af4cc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:17:59.398128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:17:59.399061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:17:59.399892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:17:59.399892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:17:59.399892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:17:59.399892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 231/471] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:17:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:17:59.407944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.434543 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.436001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.436599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.437191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.438121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:59.438628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.440628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.443027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.451146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.457439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 241/472] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:59.463530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.531596 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c0c5-821f-414b-b364-28a52ac6bc78 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.571362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.572414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.572877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.573326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.574308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:17:59.574808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.576469 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.581674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61ae1d9-cb53-475c-be60-977def3658ca None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.601378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 232/473] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:17:59.608510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:17:59.639817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.640570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.640951 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.641255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.642082 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.644289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874bd795-7ed0-40af-8868-134b449b6c0c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.670449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 242/474] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:59 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:17:59.734398 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.748402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b3a9f-c854-4e07-a259-dc5f5e5d2582 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:17:59.799182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:17:59.799823 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.800088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.800515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.801180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:17:59.802700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:17:59.808346 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7886b50-c385-454c-9089-8739cadbb25a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.808860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7886b50-c385-454c-9089-8739cadbb25a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.808860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7886b50-c385-454c-9089-8739cadbb25a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.815089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 243/475] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:17:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:17:59.823615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:17:59.824022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:17:59.824245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:17:59.833272 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 233/476] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:17:59 2022] POST /identity/v3/users => generated 249 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:17:59.867311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:17:59.868552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:17:59.923584 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:17:59.958589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 244/477] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:17:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:00.245483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:00.364142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aad5f4b-a73b-46e0-b89d-69ab884b1c84 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:00.409073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:00.409742 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:00.410038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:00.410309 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:00.410983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:00.412429 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:00.425729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.sql.core [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:18:00.425729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 01:18:00.425729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [parameters: {'id': '789b2422155f40c1862b155c8fd91b8b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 01:18:00.425729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81415) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:00.446178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return ref.to_dict() Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:00.447106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application yield resource Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise self.value Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application session.commit() Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:18:00.447995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:18:00.448815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:00.449689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:00.452059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [parameters: {'id': '789b2422155f40c1862b155c8fd91b8b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:00.453941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 14 01:18:00.460024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:00.461958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 234/478] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 14 01:18:00 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:00.462111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:00.563959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87780381-792f-4a72-82c3-382f3b64cf7b None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:00.605390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:00.606067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:00.606345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:00.606613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:00.617440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-913a560e-de53-49d3-a19a-d58184f319e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:00.622323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:00.624339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:00.626251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 245/479] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:00.632476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:00.653528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:00.762561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f618f54-1874-49e3-96cb-b82f61bca649 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:00.806692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:00.807316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:00.807592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:00.807864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:00.808538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:00.808840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:00.810259 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a048b268-37af-40c5-add5-dd091e45856e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:00.819123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 235/480] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:01.518635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcae58-69ca-41b0-9a2e-0a56b130d2d5 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:01.519713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcae58-69ca-41b0-9a2e-0a56b130d2d5 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:01.520221 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcae58-69ca-41b0-9a2e-0a56b130d2d5 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:01.521864 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 246/481] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:01.527456 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24717f-b798-42b4-bcbc-7303c929172b None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:01.527966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24717f-b798-42b4-bcbc-7303c929172b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:01.528440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c24717f-b798-42b4-bcbc-7303c929172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:01.559025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-3c24717f-b798-42b4-bcbc-7303c929172b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:01.560582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3c24717f-b798-42b4-bcbc-7303c929172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:01.643347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 236/482] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:01.953237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcae58-69ca-41b0-9a2e-0a56b130d2d5 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.005404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.056290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a57b8-7b28-4f05-a180-4d8a2e6149e9 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.056290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a57b8-7b28-4f05-a180-4d8a2e6149e9 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.056290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979a57b8-7b28-4f05-a180-4d8a2e6149e9 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.058072 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 237/483] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:02.062891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.063392 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.063878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.079241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dcae58-69ca-41b0-9a2e-0a56b130d2d5 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.114857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.115068 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:02.115500 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.115776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.116045 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.116205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:02.118961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.128064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:02.133928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:02.134842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:02.135648 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 247/484] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:02.140722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.247786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.251938 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7138c6-a09f-4805-9e4c-558f981efd80 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.260883 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 238/485] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:02.288815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.289436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.289710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.289979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.290744 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:02.291054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:02.292447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:02.301288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 248/486] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:02.306976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.445709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fa12b3-874c-47d3-b568-fe2814a900bb None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.495155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.495799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.496073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.496337 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.507552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.515059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:02.516557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:02.537022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:02.538022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:02.538881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:02.538881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:02.538881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:02.538881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 239/487] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:02.547510 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.633559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.669248 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab83fc8-30c9-4e8b-8337-0081c934ca47 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.707684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.708447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.708895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.709162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.710005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:02.710331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:02.712022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dd0dc8-49e7-42f8-aef7-7d3d233d4133 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:02.731035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 249/488] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:02.731035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3aa11-7743-4a72-af42-66ed494a0e89 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.731035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3aa11-7743-4a72-af42-66ed494a0e89 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.731035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d3aa11-7743-4a72-af42-66ed494a0e89 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.731035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 250/489] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:02.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3aa11-7743-4a72-af42-66ed494a0e89 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.761742 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.776525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64635c38-7729-4de4-b4d6-d21b130e68fe None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.809977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.810691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.810977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.811237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.821663 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-bba954d7-ae4b-4ead-a7f9-701f79e16e65 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:02.826950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:02.827841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:02.829400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 240/490] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:02.830353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.830353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.830353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.845271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d3aa11-7743-4a72-af42-66ed494a0e89 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:02.847391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:02.848577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:02.881797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:02.882451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.882718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.882980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.890663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:02.892125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:02.893561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-cb058807-22b5-4619-b7a3-27d7ff2f4b54 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:02.908959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:02.909887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:02.910854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 251/491] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:02.912059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f610c-d964-42b8-9d3d-ca90cebf1522 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:02.912059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f610c-d964-42b8-9d3d-ca90cebf1522 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:02.912059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f610c-d964-42b8-9d3d-ca90cebf1522 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:02.917583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 252/492] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:02.917583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f610c-d964-42b8-9d3d-ca90cebf1522 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:02.922712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 241/493] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:02.923348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.021677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.033109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74126391-afc2-4427-8d42-6d7f9d8a6135 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.045168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f610c-d964-42b8-9d3d-ca90cebf1522 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.082010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.082010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.082010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.082010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.082720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:03.082720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.082720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.090977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.091733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.092113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.092508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.093270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:03.093710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.096524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f613c78-5c6e-475d-b05d-57ca27eae279 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.106703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 253/494] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:03.108183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.108558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.108922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.130506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:03.131060 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 242/495] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:03.131906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.132316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:03.181934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.213834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 254/496] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:03.214926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.261423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b5bd07-bcb3-4636-aded-895af18818e4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.293404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.308785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.309523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.309866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.310234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.322162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.323831 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.326961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7721df8a-cc30-4513-9c6c-c4779e6f999a None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:03.342777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:03.343764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:03.344606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:03.344606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:03.344606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:03.344606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 243/497] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:03.345528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.373301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.374238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.374866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.375739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.447162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.458507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.460099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.460750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f94f4-6049-4275-a5af-6e66ca28fc95 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.498162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.498800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.499133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.499403 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.512515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:03.518279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:03.519326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:03.520123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 244/498] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:03.520770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.547747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.577266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 255/499] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:03.577905 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.633240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b8f80d-dd19-4ec5-b766-69ca50279ec2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.666571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.667178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.667453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.667722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.668363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:03.668661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.670084 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.678163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5fd30-0143-411f-9b36-d72c7b5c80fe None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.681509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 245/500] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:03.686784 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.698779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.713645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.714435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.714713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.715036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.725301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:03.730197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:03.731113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 256/501] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:03.732158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.765729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.788346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c2d77-b89a-43ee-885f-3553e9ffb952 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.822583 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.823105 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.823380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.823655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.824317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:03.824614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.826005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b3e3e0-3bdd-438e-8cd7-ad7710072efb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.829067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278a1599-52f2-4ed3-8d6b-602dfae9f556 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.833756 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 246/502] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:03.834944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f425c47-09c8-48c6-8e0e-f9517c362f24 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.835225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f425c47-09c8-48c6-8e0e-f9517c362f24 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.835495 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f425c47-09c8-48c6-8e0e-f9517c362f24 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.838030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 247/503] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:03.838030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f425c47-09c8-48c6-8e0e-f9517c362f24 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.865242 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.865921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.866196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.866457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.877252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.878800 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:03.892935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:03.893852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:03.894808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:03.894808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:03.894808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:03.894808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 257/504] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:03.895025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:03.909144 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.941048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f425c47-09c8-48c6-8e0e-f9517c362f24 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:03.972821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:03.973437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:03.973704 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:03.973950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:03.981065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:03.981579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:03.982527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:03.995221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:03.996335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 248/505] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/users/glance => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:03.997375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.000247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2adf94-b975-4e41-a7d8-cf65ea09cbf3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.033636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.034274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.034552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.034909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.035556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:04.035855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.037236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd6526-b8d9-4035-946a-a877be965fe6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.045163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 258/506] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:04.046325 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.046606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.046873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.064826 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:04.066183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:04.097203 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae146177-23d2-44b5-a6dc-d55a57be9674 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.124970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.131051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.131668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 259/507] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:04.131796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.131964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.132211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.132511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.143713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.145234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:04.162667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:04.163593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:04.169826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:04.169826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.169826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:04.169826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 249/508] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:04.169826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.231735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.255176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29eb8e4d-b8d0-4813-bc3a-6f6104f7f70f None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.266515 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92aabc8-b646-49bf-8744-db381864c485 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.292217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.292841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.293120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.293397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.294059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:04.294366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.295762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.299681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.300330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.300606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.300871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.301603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:04.301900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.303357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.311559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 260/509] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:04.311559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.326452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 250/510] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:04.326452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.365952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.421239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9ebd8-dbba-41fc-809a-f63a5d20aad9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.424052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ce863-77ec-48b3-b024-96f31246a36d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.454669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.455299 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.455576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.455842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.457444 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.458098 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.458377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.458648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.459282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:04.459573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.460963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.464256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.468717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.470195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.472462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 251/511] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:04.473266 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:04.484352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:04.485443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:04.491172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:04.491172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.491172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:04.491172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 261/512] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:04.491172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.572059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cd58ab-9ee5-4053-bf8c-26f8c3aabddb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.573650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.582940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c949a-701d-476d-8f50-c2c8c171c35a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.606050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.606664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.606930 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.607189 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.613681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.615177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.616451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.617003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.617298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.617545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:04.627029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:04.627892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:04.628813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 252/513] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:04.629296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.680017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.691435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.692914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.726787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc49699-14f8-4212-ba3c-ba51660264fb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.760915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.761533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.761814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.762120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.773166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.774662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:04.788794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:04.789710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:04.790671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:04.790671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:04.790671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:04.790671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 253/514] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:04.790671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.794363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 262/515] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:04.795062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.811069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.884180 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:04.893808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7af4a7-d2ad-4191-bfea-0653efc0c84b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.894245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ffa25-cc87-4d7b-b38e-1b4964cbdd25 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:04.926577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.926800 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:04.927198 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.927389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:04.927482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.927663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:04.927767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.927927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:04.928572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:04.928870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.930285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.935184 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:04.936610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:04.941703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 263/516] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:04.942518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:04.948093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:04.949151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:04.950066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 254/517] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:04.950617 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.023976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.040308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d804c9-b3e1-4212-9cb1-1b11c765f2da None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.049195 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224e0207-8039-43c7-9564-88d7c31d47a0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.076127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.076739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.077012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.077273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.077963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.078283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.079742 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.085337 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.085953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.086260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.086525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.087146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.087459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.088806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.097180 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 264/518] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.098002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.100543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 255/519] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.101326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.115206 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.201349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460df194-1c7b-49f5-b4a8-844d8da11264 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.201536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933a0e2-73d2-4f78-b14d-f1e3f6b44e85 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.234119 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.234291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.234751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.234904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.235024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.235171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.235296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.235431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.237318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.243296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.245114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.245548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.246958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:05.256799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:05.257882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 256/520] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:05.259030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:05.260980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:05.261821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:05.263558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:05.263558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:05.263558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:05.263558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 265/521] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:05.263558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.322428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.361397 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e50e1d-ad4d-4359-8231-1314786af525 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.362523 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92dbdfc-1eed-4c9f-b9e0-775738e33fc3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.395563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.395563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.395838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.396126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.396855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.397155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.397514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.398150 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.398426 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.398635 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.398726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.399398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.399705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.401114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.406492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 266/522] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.407257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.432643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 257/523] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.432643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.492361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.524869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b467cb-8816-4763-ba2e-e1069021924f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.547063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5a4b6-786f-418f-bf2e-63e04f9cb664 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.554066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.565468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.566345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.566625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.567219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.567896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.569370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.585299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.585926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.586234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.586484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.587224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.587529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.589044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.594554 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 267/524] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:18:05 2022] POST /identity/v3/users => generated 247 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:05.595892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.610429 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 258/525] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.611413 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.732153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a873e-3d70-4b8f-b89c-088d052e7ae0 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.754850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d4c5c6-3c9d-40ac-a4a0-dd98ec829b0e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.768840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.773004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.773889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.774338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.774767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.775555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:05.776006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.778260 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.794465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 268/526] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:05.795596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:05.802293 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.803035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.803412 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.803776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.850871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:05.883516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:05.885004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:05.928516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257cf0a-5040-453a-9d22-74c6c1216e7a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:05.969918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:05.970394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:05.970673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:05.970942 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:05.997402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 259/527] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:05.998611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.010782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.056428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.058433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.104408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e23f5c-328a-4426-b28e-aa672999edd8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.119212 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.144818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.145611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.145979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.146410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.154015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.155831 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:06.168963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:06.169881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:06.170765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 260/528] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:06.171947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.197043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 269/529] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:06.197043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.251238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.285922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7aa654-af6d-4896-afad-343e8d869e8a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.308063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca987f2-7a64-4048-b5fd-26743c00b1f7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.324975 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.325640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.325910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.326633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.339237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.340722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.351435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.352582 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.353037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.353478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.354368 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:06.354845 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.356467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:06.357481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:06.358531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:06.359375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:06.359375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:06.359375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:06.359375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 261/530] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:06.359607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.377011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 270/531] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:06.378189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.381846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.474194 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d8efc2-ee6f-4a2b-aac7-73604d00a380 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.490064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc4c76-1b2f-42d4-9cb1-eb19454c5d20 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.497577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.511385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.512024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.512296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.512564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.524319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.525626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.529431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.530357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.530836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.531343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.532241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:06.532748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.534339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:06.540902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:06.541755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:06.542634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:06.542634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:06.542634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:06.546741 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 262/532] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:06.547697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.554194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 271/533] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:06.555950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.632499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.670170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad1141e-63b7-4bfa-9d58-4b0261538bf5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.677868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31452fa-df85-4e05-ad7c-03b370de5bdf None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.707116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.707728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.708001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.708266 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.709098 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=0aad071f42b547938aac8b6063e4a1bd) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:06.709433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.711026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.717800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 263/534] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/role_assignments?user.id=0aad071f42b547938aac8b6063e4a1bd&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:06.718463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.718695 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.719352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.719633 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.719897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.720724 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:06.721158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.723642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:06.741027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 272/535] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:06.747427 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:06.789155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:06.900795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8251ce6e-e326-4de5-b499-187e30a5f4d1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.959245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1fca27-24a7-479c-8590-f7109ce71360 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:06.963968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:06.964609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:06.964880 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:06.965147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:06.980811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:06.982303 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:07.002644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:07.003612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:07.004427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 264/536] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:07.004886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:07.013204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:07.030697 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:07.031352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.031631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.031896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.039609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:07.050341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:07.075478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:07.076376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:07.077334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:07.085862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 273/537] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:07.089527 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:07.158561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59823449-201b-4abe-b5d6-a1ffc42f25fe None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:07.162168 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:07.198056 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:07.199066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.199483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.199894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.200891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:07.201357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:07.203086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.223183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 265/538] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:07.229233 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:07.260931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc5ef4-f912-40fc-9f15-bed77b7b2cf6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:07.297729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:07.298643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.299144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.299581 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.300506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:07.300964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:07.302732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.321851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 274/539] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:07.323356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:07.374665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:07.437162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f760102c-a9d7-408a-9e19-072845a052f3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:07.462359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f5cdd8-8f86-466f-b7f5-d06ea8f8e7f7 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:07.474875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:07.475734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.476152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.476676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.477546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cb571f4363c34886b8763cab9310459f) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:07.478022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:07.479602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c8cbbb-17d5-46e2-9be3-4b3e2d8b0c34 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.489852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 275/540] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:07 2022] GET /identity/v3/role_assignments?user.id=cb571f4363c34886b8763cab9310459f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:07.541092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:07.541714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.541999 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.542266 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.543002 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=1f5db3c037944dfbb2e4dd1bd8fe24b3) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:07.543308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:07.544778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146da9e-fe2e-4310-a0e8-e941be2247ff None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:07.554868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 266/541] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:07 2022] GET /identity/v3/role_assignments?user.id=1f5db3c037944dfbb2e4dd1bd8fe24b3&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:07.769118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb80e6c-6e50-4dbc-8e7e-494cf4de010b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.770155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb80e6c-6e50-4dbc-8e7e-494cf4de010b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.770655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb80e6c-6e50-4dbc-8e7e-494cf4de010b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.772605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 276/542] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:07.778928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30979b-2443-4a64-8bbf-bcf5c7230836 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:07.779223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30979b-2443-4a64-8bbf-bcf5c7230836 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:07.779476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c30979b-2443-4a64-8bbf-bcf5c7230836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:07.807024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-8c30979b-2443-4a64-8bbf-bcf5c7230836 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:07.808297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-8c30979b-2443-4a64-8bbf-bcf5c7230836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:07.829565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:07.894909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 267/543] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:08.171158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bb4b-cf69-42a3-a2ec-9e40f81af860 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:08.171530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bb4b-cf69-42a3-a2ec-9e40f81af860 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:08.171703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a52bb4b-cf69-42a3-a2ec-9e40f81af860 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:08.173491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 277/544] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:08.179510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e03f-004a-4190-833b-d356304174be None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:08.179967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e03f-004a-4190-833b-d356304174be None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:08.180073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e03f-004a-4190-833b-d356304174be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:08.202360 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-6a66e03f-004a-4190-833b-d356304174be None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:08.203562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-6a66e03f-004a-4190-833b-d356304174be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:08.297751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 268/545] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:08.321635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52bb4b-cf69-42a3-a2ec-9e40f81af860 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:08.461006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a52bb4b-cf69-42a3-a2ec-9e40f81af860 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:08.471103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:08.516182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:08.516831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:08.517100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:08.517362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:08.531900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:08.536811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:08.537677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:08.538524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 278/546] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:08.545818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66e03f-004a-4190-833b-d356304174be None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:08.673667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:08.699284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66e03f-004a-4190-833b-d356304174be None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:08.740254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:08.740801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:08.741075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:08.741346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:08.742080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:08.742381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:08.743916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:08.758927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 269/547] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:08.764544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:08.830023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:08.967841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05626ff2-2fa1-4dcf-b3e8-800b81d15045 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:08.991710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b1e57c-4307-4ca6-bd59-4633551546df None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.041947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.042537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.042810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.043077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.055822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.057368 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:09.074021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:09.074981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:09.075830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:09.075830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:09.075830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:09.075830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 279/548] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:09.087175 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.091055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.091936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.092508 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.093941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.116689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:09.128487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d5c0659a-4091-409e-8c9e-273636c191f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:09.133328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:09.134770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:09.135733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:09.142109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 270/549] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:09.152323 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6ead-3746-4aaf-8394-1d814e0dc06d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.152862 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6ead-3746-4aaf-8394-1d814e0dc06d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.153332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc6ead-3746-4aaf-8394-1d814e0dc06d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.155386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 271/550] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:09.156274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc6ead-3746-4aaf-8394-1d814e0dc06d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.160634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.235213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2e273e-84a6-4d14-bdcd-d7110c9dc3d2 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.279672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.280862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.281708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.282479 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.283326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:09.283887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.285273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cc6ead-3746-4aaf-8394-1d814e0dc06d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.286214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d656457f-918d-4ea5-ae96-7740e6e4cabc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.302735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 280/551] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:09.304655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.305081 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.305529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.322075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.322868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.323239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.323609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.324474 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:09.324922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.326458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.329835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:09.331204 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:09.336040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 272/552] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:09.337012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.404365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.421449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 281/553] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:09.422599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.433733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.464148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d46d3c6-b265-4c07-bfbd-ec9d00eebd7a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.536154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a723c4eb-a301-4f4f-8085-b474fd031b52 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.539750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.540157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.540533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.540901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.557344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.559053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.575316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.576414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.576958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.577439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:09.581639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:09.582705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:09.583622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:09.590613 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 273/554] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:09.591582 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.593250 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.595723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:09.610991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:09.612038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:09.613023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:09.613023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:09.613023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:09.615174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 282/555] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:09.623118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.760410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506ea02-36f3-4428-b43e-4cbecc7d82cb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.763284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.790441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:09.804963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.806018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.806496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.806960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.807894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:09.808393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.810081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.823680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc844d2a-6626-46f7-9442-bd9ceb877f0b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.835573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 274/556] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:09.835573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.861933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.862732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.863006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.863322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.863957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:09.864302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:09.865922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec968b63-26a9-4622-a1b0-e5f2c84f30c3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:09.878488 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 283/557] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:09.883896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff7f07-4ed4-4270-a4e0-5f2b5781fa38 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.884188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff7f07-4ed4-4270-a4e0-5f2b5781fa38 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.884460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff7f07-4ed4-4270-a4e0-5f2b5781fa38 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.885834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 284/558] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:09.886450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff7f07-4ed4-4270-a4e0-5f2b5781fa38 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:09.940627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2693da53-b6af-42ed-b32e-f021a2830b4e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:09.974670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:09.975579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:09.976131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:09.976569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:09.987204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:09.992225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:09.993836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:09.994754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 275/559] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:09.995447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.005078 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff7f07-4ed4-4270-a4e0-5f2b5781fa38 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.031111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.046214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.049844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.051223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.051495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.051757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.062573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.063990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-6646883e-0be1-4efb-b06a-e479ed427c9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:10.081508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:10.082717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:10.083809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:10.083809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:10.083809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:10.085440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 285/560] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:10.086783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.087054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.087320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.106222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:10.107565 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:10.113281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e74ede3-936b-4c7a-98ea-111a3aa6ed86 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.146349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.146970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.147239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.147500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.154338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.155777 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:10.167009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:10.167987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 276/561] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:10.168981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.180072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 286/562] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:10.181240 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.270892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bfdf9d-d567-41bf-a2b9-f8c062f04405 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.274618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.295187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba43f26-9e4b-4745-af2f-d9bfb590ae99 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.297378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.307875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.308487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.308768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.309062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.309760 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:10.310093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.311502 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8694cff9-e464-4f53-84be-a3766fcd6e4e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.319849 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 277/563] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:10.321022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa0c3-36bb-42ae-9cba-2105e44e91c8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.321302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa0c3-36bb-42ae-9cba-2105e44e91c8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.321570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59aa0c3-36bb-42ae-9cba-2105e44e91c8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.326105 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 278/564] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:10.326105 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa0c3-36bb-42ae-9cba-2105e44e91c8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.341049 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.341667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.341943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.342384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.343027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:10.343535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.344777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.361788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 287/565] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:10.362740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.428004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59aa0c3-36bb-42ae-9cba-2105e44e91c8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.461325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.461933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.462238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.462397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7be87b9-66c1-4a21-bc28-b20511bf8c03 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.462516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.463221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:10.463522 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.464962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bef558-0a85-4314-a545-993042ad65dd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.481610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 279/566] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:10.482812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.483090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.483355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.494773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.495404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.495678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.495943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.501007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:10.502237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:10.506241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.507675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.519404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:10.522252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:10.523185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:10.524065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:10.524065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:10.524065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:10.524065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 288/567] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:10.524293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.539411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.573609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 280/568] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:10.574529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.629661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eae22-f61d-43bc-9336-14c7e69511ae None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.665306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.666301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.666827 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.667119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.672619 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32387d82-7a74-4788-8ade-f2f9a6a09018 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.674063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.675647 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:10.688740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:10.689978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:10.691011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 289/569] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:10.692155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.713813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.714756 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.715037 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.715331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.726299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:10.731158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:10.732109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 281/570] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:10.734956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.753035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.778259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.807355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9f3fd-a155-4874-858d-906391632b6b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.840254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c68aabd-fadf-45ba-97c8-93c02d0d5a5f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.844061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.844677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.844954 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.845224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.873580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:10.874211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:10.874481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:10.874743 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:10.875384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:10.875682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.877051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.888041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 282/571] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:10.888801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:10.914559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:10.916002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:10.960660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:10.985500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069e7b8-7f89-4f01-9773-b4e5cc538513 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:10.988215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.014047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 290/572] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:11.015002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.019144 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.019820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.020096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.020363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.021049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.021349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.022808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.034605 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 283/573] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.035382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.119579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c1f181-6ddf-4164-b846-571358c43091 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.138414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958852e3-4abd-4c47-9ce4-b364a9db9fe2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.153669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.154298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.154572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.154816 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.155481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.155774 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.157141 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.164929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 291/574] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.165702 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.173469 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.174124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.174391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.174652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.176581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.186649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:11.191812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:11.192805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 284/575] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:11.193690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.207930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.272972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae31a87-bbab-4f91-b5ef-b2a080dbb56d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.294346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbe0b60-0d44-49c7-8a2b-e6846e4a408f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.305117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.305738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.306021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.306307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.313926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.315785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.326415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:11.326906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:11.327803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:11.328747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:11.329264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.329264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.329264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.329458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 292/576] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:11.329458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.339895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.341331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:11.355585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:11.356518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:11.357409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:11.357409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:11.357409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:11.357409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 285/577] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:11.357409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.401868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.426549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.432146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e17ff0-c5be-4dd3-9325-9d9eb0dc687e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.453551 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71665bd9-5dbf-430d-8717-2980246ac805 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.465678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.466354 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.466630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.466899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.481546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=6f6158bf6948476a9244d0c28e1c8404, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.483041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.486300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.486921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.487234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.487554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.496335 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 293/578] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:11 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:11.497130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.506557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.508263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:11.525350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:11.526460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:11.527372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:11.527372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:11.527372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:11.527372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 286/579] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:11.527615 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.610483 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee2f69e-40fa-4878-965d-740293868382 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.636469 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.640992 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a3a077-7cbe-41cc-a17b-ddc9503bbb6e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.648984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.650278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.650627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.650960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.651708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.652481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.654265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.663564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 294/580] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.664529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.680171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.687819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.688389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.688671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.688922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.690065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.690360 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.691904 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.710196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 287/581] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.711164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.778801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376c1d62-61e5-4bbc-a9ec-ae0e1496a00a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.807475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522a6dd9-1747-4418-bf35-a902ffacde87 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.815044 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.815673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.815953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.816213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.816951 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.817251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.818743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.835775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 295/582] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.835775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.846459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.847104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.847395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.847659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.848287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:11.848583 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.849959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:11.861564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 288/583] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:11.862767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:11.881070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.909526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:11.945068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60102e06-5c52-4398-a5c8-968f7246d589 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.966192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aa0591-5a5d-40df-a3fe-c243c3b4c0ac None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:11.981666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:11.982410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:11.982955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:11.983284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:11.994959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:11.996425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.000376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.001012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.001281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.001545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:12.011574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.012507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:12.018566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:12.018566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.018566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:12.018734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.018877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 296/584] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.018952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.019023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:12.033650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.034856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:12.035734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:12.035734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.035734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:12.035734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 289/585] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.035955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.107682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.133771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffcc632-bc26-49f7-aab9-3e32470ce623 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.152953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f22633c-fd8f-4b4d-9e74-f83f1823d5af None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.153622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.169038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.169888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.170332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.170766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.177732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.179230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:12.192231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.193161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:12.194068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 297/586] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.194953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.201138 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.202023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.202023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.202313 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.210428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.211944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:12.223665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.224647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 290/587] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.225554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.306960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902da245-c826-40ec-abe6-864c5ed5dc68 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.327962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e1089d-0449-463c-bc5e-9e7fcf91eba4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.341103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.342013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.342404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.342764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.343550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:12.344875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.346269 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.362572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 298/588] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:12.362572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.368168 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.378650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.379353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.379687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.379953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.380593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:12.380888 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.382330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.394496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 291/589] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:12.395297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.413335 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.478680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a2011-817b-4c43-9da6-1bda123a9c5b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.498066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f6bbac-2b6b-40b4-a5c8-bc0d358b6353 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.517316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.517919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.518216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.518477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.519238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:12.519532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.520958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.537567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 299/590] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:12.537567 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.539599 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.540214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.540485 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.540748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.541481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:12.541779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.543242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.565268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 292/591] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:12.565268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.613342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.646096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.649637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53433daf-8e64-4b65-8552-9b1456d68663 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.666512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facad142-6f66-4ff5-ab15-bf4ad4da45aa None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.686431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.687313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.687421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.687594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.696042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.697655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.703315 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.704463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.704930 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.705407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:12.709192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.710155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:12.711089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:12.713698 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.715263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 300/592] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.715388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.715938 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:12.726938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.727851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:12.728788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 293/593] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.729297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.824117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466809a8-f60c-41fe-9108-298dc3091216 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.831390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:12.832469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81e9dee-d8c0-4966-9f00-9f343697e5d7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:12.859123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.859753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.860028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.860293 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.871043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.872483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.872780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:12.873529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:12.874104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:12.874536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:12.875461 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=03da2d917c354a389dda2242ac5d4297) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:12.875922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:12.877486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:12.883316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 301/594] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/role_assignments?user.id=03da2d917c354a389dda2242ac5d4297&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:12.884221 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:12.886249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:12.887179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:12.888122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:12.888122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:12.888122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:12.888122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 294/595] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:12.888356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:12.925575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.004231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2653ef3-cef6-4763-a1ac-4d841ab99a97 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.025024 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccedc05-dd96-453f-b1ec-0a0b3b2e1e0f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.044729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.045367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.045641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.045907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.046768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:13.047086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.048785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.062675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 295/596] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:13.069848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.070752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.071174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.071533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.071774 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.072912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:13.073359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.075005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.095486 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.124478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 302/597] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:12 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:13.127275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.195115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85068e-18e9-44db-b5fe-94159939d46c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.222251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.239520 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.240686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.241497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.242167 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.242975 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:13.243510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.243745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ed50a6-4d0a-45f4-9d78-d9f74aa10b7b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.245437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.269365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 296/598] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:13.269365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.293848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.294546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.294820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.295085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/0aad071f42b547938aac8b6063e4a1bd/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.321464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=0aad071f42b547938aac8b6063e4a1bd, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.323305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.342631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 303/599] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:13 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/0aad071f42b547938aac8b6063e4a1bd/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:13.343814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.394884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.438354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fce7f9-8b90-4cc9-afc1-910513cef41b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.472098 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1533c983-d21a-4993-8ec3-758b7e17643a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.510805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.518073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.519879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.520400 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.521245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.531685 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.533155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.533825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.533937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.534179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.534413 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:13.548784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:13.549769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:13.550797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:13.553496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 304/600] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:13.560442 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.561948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.565033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:13.595278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:13.596289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:13.597230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:13.597230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:13.597230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:13.597230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 297/601] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:13.602642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.708021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8453f6-65d4-4c42-b904-665a178cae15 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.714408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1cb0d0-3c65-4643-a469-7ab72a29b38a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.746113 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.753249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.753838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.754289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.754869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.755680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:13.755984 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.757427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.762902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.763522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.763844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.764111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.764911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:13.765212 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.766677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9b5eb7-f343-4f62-ba20-9d661721a539 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.770936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 298/602] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:13.778102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.789448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 305/603] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:13.790963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea441bf-0768-4083-8381-a0d6d1c10d3d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.791314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea441bf-0768-4083-8381-a0d6d1c10d3d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.791791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea441bf-0768-4083-8381-a0d6d1c10d3d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.794361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 306/604] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:13.799420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea441bf-0768-4083-8381-a0d6d1c10d3d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:13.860369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:13.906079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5fffa-b81d-43f8-8d52-dbf928bc4ad9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.917784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea441bf-0768-4083-8381-a0d6d1c10d3d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:13.954904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.955662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:13.955824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.956274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.956657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/1f5db3c037944dfbb2e4dd1bd8fe24b3/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.956907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.957391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.957810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.967070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.969245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.973020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=1f5db3c037944dfbb2e4dd1bd8fe24b3, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:13.974568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8807adee-f6e9-463d-a350-da1499e71941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:13.983668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:13.984597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:13.985471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:13.988047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 299/605] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:13.989682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:13.990177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:13.990640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:13.997297 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 307/606] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:13 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/1f5db3c037944dfbb2e4dd1bd8fe24b3/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:13.998323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:14.011575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:14.013427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:14.068046 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:14.092757 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 300/607] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:14.116854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db581b4-856e-4d6a-b74f-e2590abdf42c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:14.118707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:14.155632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:14.156255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:14.156526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:14.156797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:14.157507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:14.157882 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:14.159410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:14.173056 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 308/608] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:14.180089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:14.307210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2d148-3739-452f-842c-5e394552e4c5 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:14.344149 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:14.344732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:14.345001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:14.345268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cb571f4363c34886b8763cab9310459f/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:14.362049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=cb571f4363c34886b8763cab9310459f, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:14.363700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:14.388484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 301/609] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:14 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cb571f4363c34886b8763cab9310459f/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:14.465791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:14.579673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552e9a-da1c-4f50-9739-e6acf389c5e7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:14.614999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:14.615645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:14.615917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:14.616227 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:14.626767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:14.632705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:14.633607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:14.634458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 309/610] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:14.639853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:14.687287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:14.762949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7d497f-6a20-4a80-a25b-bd18a9dbd081 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:14.808573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:14.809185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:14.809460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:14.809728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:14.810432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:14.810732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:14.812176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:14.825421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 302/611] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:14.831238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:14.864032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:14.959948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e188cd-3b3c-4f25-8335-0e5330e014df None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:14.997272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:14.997906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:14.998201 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:14.998467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.012205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:15.013664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:15.032476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:15.033451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:15.034414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:15.034414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:15.034414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:15.037538 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 310/612] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:15.047098 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.188250 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:15.198614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04288ae3-a3a5-4937-8922-5239e4244a7f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:15.242820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:15.243351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.243624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.243886 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.244530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:15.244815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:15.246204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:15.263059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 303/613] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:15.268753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.368194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:15.404136 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172e2316-64c3-46f7-b6bd-af42498cee9b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:15.445405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:15.446053 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.446330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.446597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.455126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:15.456597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-59dd62fc-ae0c-4ad7-9c75-e9be64ecfa86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:15.472609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:15.473629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:15.474599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:15.478089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 311/614] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:15.483205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.622704 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed4ab0-fe47-414e-b58c-d6074fc5b1a4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:15.658143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:15.658778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.659039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.659270 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.660044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:15.660343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:15.661782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:15.677627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c904c1-ead5-476a-9ec6-30ac2e421e96 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.677926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c904c1-ead5-476a-9ec6-30ac2e421e96 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.678187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c904c1-ead5-476a-9ec6-30ac2e421e96 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.679522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 312/615] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:15.684790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b431e-02f6-48ac-8ef9-0e50a477e6ec None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.685067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b431e-02f6-48ac-8ef9-0e50a477e6ec None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.685330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b431e-02f6-48ac-8ef9-0e50a477e6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.688036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 304/616] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:15.695590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:15.716991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-503b431e-02f6-48ac-8ef9-0e50a477e6ec None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:15.718675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-503b431e-02f6-48ac-8ef9-0e50a477e6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:15.722903 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:15.851484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6104a7-ac6b-4be8-87b1-d1576db03e3d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:15.864440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 313/617] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:15.865726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17dc3e-9dd0-4468-89ab-226d3e03e835 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.874529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17dc3e-9dd0-4468-89ab-226d3e03e835 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.874771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b17dc3e-9dd0-4468-89ab-226d3e03e835 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.876484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 314/618] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:15.881587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.881858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.882250 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.891201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:15.891858 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:15.892130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:15.892398 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:15.915427 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:15.916622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:16.020705 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.022565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.030314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.041026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.042940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 315/619] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:18:16.152834 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 305/620] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 14 01:18:16.159131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.178462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.300681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9459e056-0ed9-4fff-8280-649e64b2d1cd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.312908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.339573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.340567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.340843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.341111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.352309 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:16.357251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:16.358164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:16.359145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 306/621] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:16.365921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.378005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680021-29e5-4ce8-85c5-62e424dca279 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.421406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.422363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.422809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.423261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.434356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.436035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:16.452101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:16.453062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:16.453960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:16.453960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:16.453960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:16.456043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 316/622] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:16.457080 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.499608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257aec5c-df1a-4249-9795-a2eff93a2b23 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.513773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.552503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.553121 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.553493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.553796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.554744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:16.555047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.557038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.572052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 307/623] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:16.574387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.582920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299c133e-abaa-4d10-9c3d-f38ce3415dd0 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.611669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.631477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.632119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.632404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.632660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.647189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:16.654033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:16.655032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:16.655894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 317/624] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:16.656629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.693747 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502fe40-d8c2-43c2-9965-1f5bee0bf65d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.732299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.733317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.733794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.734288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.735124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:16.735628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.737193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.750201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 308/625] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:16.751671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.789573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9b785-b32d-4152-8338-f35d77578ea1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.797665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.830840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.831498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.831768 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.832196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.844330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.845709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:16.860534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:16.861520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:16.862472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:16.862472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:16.862472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:16.862472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 318/626] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:16.862718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.864463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:16.873738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ccccef-fac7-4437-8320-f4e7c414f88d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:16.908613 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:16.909213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.909601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.909874 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.910561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:16.910861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:16.912250 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ff942d-cbbd-4b5f-8825-99f9a764e910 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:16.920597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 309/627] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:16.921744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0036d-f166-4991-b4e0-b82fb0daa59f None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:16.922049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0036d-f166-4991-b4e0-b82fb0daa59f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:16.922320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0036d-f166-4991-b4e0-b82fb0daa59f None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:16.923639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 310/628] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:16.924254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0036d-f166-4991-b4e0-b82fb0daa59f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:16.981040 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b2eea-f44e-4622-b9f4-93e4e32b9060 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.016782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.017755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.018056 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.018347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.026504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.027964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.035085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0036d-f166-4991-b4e0-b82fb0daa59f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:17.040339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:17.041586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 319/629] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:17.042721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.067061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.081689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.082375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.082650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.082919 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.083618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:17.083982 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.085372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3e2f04-ed43-41c4-ad48-cd75dac3a8b5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.098600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 311/630] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:17.104041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.104590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.105064 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.127821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:17.129161 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:17.139040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.155043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b18378b-46ae-4eea-8768-a7d1d8a84164 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.187898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.188526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.188799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.189064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.194572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 312/631] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:17.208567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.208674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.208674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:17.216352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:17.217457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:17.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:17.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:17.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:17.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 320/632] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:17.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.291631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.298916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d109dd7-8e27-4fd4-8856-fc0f2c368872 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.324512 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0fb3a-b985-452b-84e9-e9405307650b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.335403 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.336124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.336404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.336673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.337523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:17.338217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.340177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5cbdc-313e-4715-85e4-6f1842f77aae None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 313/633] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36d96e-aaa8-4265-8c86-6c6066aaccc3 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36d96e-aaa8-4265-8c86-6c6066aaccc3 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b36d96e-aaa8-4265-8c86-6c6066aaccc3 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 314/634] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:17.359335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36d96e-aaa8-4265-8c86-6c6066aaccc3 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.362863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.363496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.363775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.364033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.364860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.374178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.375892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:17.388358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:17.389748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:17.390746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:17.394361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 321/635] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:17.394947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.469675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b36d96e-aaa8-4265-8c86-6c6066aaccc3 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.500311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c02e1b-2044-4dd0-b894-878ec34021d5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.502169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.502769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.503037 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.503349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.503975 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:17.504268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.505625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9c1177-7c58-4438-afb5-f0611b179dd7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.517229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 315/636] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:17.518591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.519041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.519462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.530727 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.536754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.537388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.537664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.537935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.538072 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:17.538703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:17.539004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.539284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:17.540478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.546444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 322/637] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:17.547163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.611367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.622055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 316/638] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:17.622317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.672739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3bfed1-0288-48c7-8eee-aedeca43f25a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.712694 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.713321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.713594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.713930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.714894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:17.715065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.716452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.743126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 323/639] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:17.744123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.774219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2e181-e548-4e19-b8bf-ae5399aab3ea None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.807214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.821885 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.822537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.822810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.823090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.834639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:17.839638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:17.840782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:17.841909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:17.844397 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 317/640] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:17.845365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.871139 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a65dee-703a-41dc-9ed7-e488e7af98b4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:17.913022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:17.913664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:17.913936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:17.914231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:17.920050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:17.921974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:17.923557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:17.938615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:17.940077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:17.941006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 324/641] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:17.942309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:17.974958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f15ff4-cc28-4156-abee-549fb9ea2703 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.008827 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.009494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.009769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.010079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.021455 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.022946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:18.037941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:18.039035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:18.040525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:18.040525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:18.040525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:18.040525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 318/642] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:18.040525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.057109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcd888-a9be-467e-b023-9808449226fb None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.071213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.098496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.099618 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.099896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.100426 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.101160 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:18.101701 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.103272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.114167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 325/643] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:18.115131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.150951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c814c1f-3eba-4563-a455-dd90068546de None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.158699 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.188257 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.188893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.189172 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.189795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.203583 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:18.209232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:18.210221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:18.211124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 319/644] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:18.212681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.232945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991622e-e898-49b5-80cf-5ba2d92711a3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.277403 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.277669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.277885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.278187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.278916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:18.279220 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.281392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.299762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 326/645] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:18.300609 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.322818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf1c42e-1d20-4005-a9c3-83bcbe19c4ad None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.333074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.358696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.359299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.359568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.359830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.360470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:18.360765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.362160 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.374526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 320/646] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:18.379838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.407560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.417843 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdc98a-f9a7-4baa-89e1-fe26c9df7568 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.453770 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.454444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.454724 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.454991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.466319 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.467846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.481456 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b117f23-fb4c-4d9e-b6a9-de0506be0a64 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:18.486814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:18.487743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:18.488632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:18.488632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:18.488632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:18.488632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 327/647] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:18.488861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.519407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.520013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.520287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.520552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.521222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:18.521528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.522932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.535493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 321/648] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:18.535493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.584315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.602492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af1a-5dfa-4a47-8fe4-7d636a504d93 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.636302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.639052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.639669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.639945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.640397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.650430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2f48f8-71b5-4419-9c1a-5da300f398e6 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.686517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.687151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.687424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.687693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.694820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.696282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:18.707339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:18.708231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 322/649] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:18.709150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.717117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.718629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.805265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.823414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c56308-3e1e-4dc4-bceb-3ddedc4982b0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.824132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 328/650] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:18.827719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.858400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.859027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.859201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.859472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.860117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:18.860415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.861782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:18.873509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 323/651] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:18.873509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:18.886512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:18.937531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f679ac0a-d59c-4fd0-802f-d0672d4bd966 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.973886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:18.974803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:18.975138 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:18.975582 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:18.984783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35310858-8e8a-4044-8af5-d07161735f96 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:18.986951 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:18.988680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:19.004406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:19.005339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:19.006626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:19.006626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:19.006626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:19.006626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 329/652] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:19.006626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.028176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.028787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.029057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.029321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.030029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:19.030330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.032488 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.044992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.050290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 324/653] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:19.051035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.121955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.129404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f1558-14b0-41c2-89dc-1ee18f3c2874 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.157202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd69d61a-29ff-4e4c-a1cb-336b64442eaa None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.162425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.163054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.163325 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.163590 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.175137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.176762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.191679 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:19.191921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:19.192804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:19.193692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:19.193692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:19.193692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:19.193868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.193868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.193868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.197614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 330/654] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:19.198465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.201325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.202829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:19.214190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:19.215123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:19.216054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 325/655] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:19.216642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.291574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.305930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a25e22c-5506-46b4-9f3a-2d0209ecda89 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.326118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f65f1e-d3a7-413a-b6f1-df04278830b8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.340165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.340773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.341041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.341307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.341943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:19.342262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.343634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.354353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.355598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 331/656] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:19.356236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.367330 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.367995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.368271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.368544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/03da2d917c354a389dda2242ac5d4297/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.384247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=03da2d917c354a389dda2242ac5d4297, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.385677 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.402030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 326/657] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:19 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/03da2d917c354a389dda2242ac5d4297/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:19.402030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.504700 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfe2f4f-0d7b-46d4-941d-bcc3ca7ac5fa None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.519017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a8cc3-2883-4d7b-b0f0-364e7654e9ce None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.549270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.549507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.549683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.550152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.550814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:19.551111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.552507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.567067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 332/658] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:19.568372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.568683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.574091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.588323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.596313 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 327/659] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:19.598219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.638162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.704334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764f845-312d-4822-8aae-cd7b4c03fabc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.714239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca3cad7-3767-4286-b2b3-d0981ee7cbb5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.753372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.754063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.754341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.754608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.759618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.760645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.761139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.761636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.764536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.766484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.770372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.772046 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:19.780074 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:19.781031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:19.781955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 333/660] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:19.783032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:19.787993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:19.788878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:19.789864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:19.792070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 328/661] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:19.793048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:19.869182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.903886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90341fa-69e1-426c-8c46-b3433bbad3a2 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.911041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:19.918580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86179296-5a04-497a-a5d8-d5b299851906 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:19.941860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.942533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.942956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.943231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.953776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:19.954396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:19.954668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:19.954935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:19.955660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:19.955962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:19.957447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:19.980050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 329/662] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:19.980050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.025363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.026867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c5d20b-1f2b-42a2-a931-764e59615bed None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.097631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b17b711-24a6-43a6-9ba8-c51f8d274c78 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.108119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.141001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.141860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.142216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.142909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.144066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:20.144436 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.146791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.154085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 334/663] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:20.155476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48c48-3469-4dc2-82bc-236d84f3a075 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.155828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48c48-3469-4dc2-82bc-236d84f3a075 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.156170 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48c48-3469-4dc2-82bc-236d84f3a075 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.157765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 335/664] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:20.158553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48c48-3469-4dc2-82bc-236d84f3a075 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.170805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 330/665] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:20.170805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.190177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.273792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48c48-3469-4dc2-82bc-236d84f3a075 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.298367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9fd408-8571-4455-a80a-cef390f59717 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.311488 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.312190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.312465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.312733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.335688 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.337341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.337830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.338300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.351215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.352745 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.364844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-35dd7a3d-7e85-43a2-a4d8-78a5cccf61b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:20.369634 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:20.370727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:20.371614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:20.371614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:20.371614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:20.375902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 331/666] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:20.377158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.377766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.378210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.399525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:20.401247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:20.414332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.415895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.478075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.489233 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 332/667] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:20.490310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.548792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 336/668] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:20.549674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.623511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.631143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1513af6-ec2e-4fd0-8be3-6b856d601fda None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.662242 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53887623-7f5d-495a-b31d-b60f8f06de3e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.674542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.675129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.675681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.675957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.676692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=0aad071f42b547938aac8b6063e4a1bd) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:20.677255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.678990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.686050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 333/669] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/role_assignments?user.id=0aad071f42b547938aac8b6063e4a1bd&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:20.687014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.703827 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.704914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.705464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.705948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.706909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:20.707425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.709298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.723177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 337/670] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:20.729101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.753934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.826421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523466e-b306-4f70-a54e-d76d49194089 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.839805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5697f0-00a0-482c-a37a-f1b96514d930 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:20.866598 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.867476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.867591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.867852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.871230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:20.880086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:20.881083 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.881345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:20.881813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:20.882329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:20.882695 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.902025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:20.902025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:20.902351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:20.903219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:20.904103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:20.904103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:20.904103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:20.904103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 334/671] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:20.904103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:20.917572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:20.918611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:20.919533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 338/672] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:20.920849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.026101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4240492-75b8-48c0-8f64-b55dd8ffd7bd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.040943 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.054332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974acce-1927-4474-a5e1-b9e4c970e49d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.073346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.073346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.073346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.073346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.085465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:21.093081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:21.094016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:21.095526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 335/673] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:21.096227 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.102253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.102876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.103147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.103411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.104055 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.104347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.105732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.118501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 339/674] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:21.119802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.181519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.221702 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3eecd0-2944-49b2-98b7-b9ff272ae3bd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.249920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9c1bc-a87c-49d3-8b97-6e2232f93033 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.264285 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.264908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.265183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.265446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.266550 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.266861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.268684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.286134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 336/675] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:21.287294 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.292234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.293121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.293585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.294073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.294926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.295410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.296979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.311051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 340/676] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:21.312265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.326084 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.412585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf690299-cb0e-4690-be26-56a2e4339879 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.423395 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c0b320-71c7-4469-b6b8-323f749fd09d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.437275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.455466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.456510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.457024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.457577 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.464975 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.465834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.466477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.466752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.467019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.467596 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.467772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=1f5db3c037944dfbb2e4dd1bd8fe24b3) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.468041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.469602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:21.480207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:21.481148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:21.482089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:21.482533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 341/677] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/role_assignments?user.id=1f5db3c037944dfbb2e4dd1bd8fe24b3&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:21.483351 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.484764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 337/678] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:21.494378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.604741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea639296-7e9c-4f76-83e1-e3f70a40712a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.645865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.646545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.646818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.647083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.661809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.663818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.682352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-8837899e-d0e5-4fb9-8724-c862c738cfa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:21.683976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:21.684884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:21.685791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:21.685791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:21.685791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:21.686364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 342/679] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:21.687397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d703c-acc7-4b32-ad77-d4cbf251b906 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.687700 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d703c-acc7-4b32-ad77-d4cbf251b906 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.688000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d703c-acc7-4b32-ad77-d4cbf251b906 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.688723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea4c289-c770-48a9-9c0e-a8e1426361f7 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.689891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 343/680] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:21.694348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d703c-acc7-4b32-ad77-d4cbf251b906 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.726668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.727296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.727574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.727843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.728546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.728860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.730470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54648800-0211-463b-bfe8-47fd9a51ddb0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.734389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.744217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 338/681] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:21.745633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.745976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.746271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.767966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:21.769621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:21.831085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d703c-acc7-4b32-ad77-d4cbf251b906 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.855319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 339/682] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:21.856292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.866790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:21.867636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:21.868071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:21.868480 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:21.869261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:21.869724 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:21.871256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:21.887143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 344/683] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:21.892310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:21.962385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:21.971947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de1520-1bf5-4c62-8d4f-f186d16e614b None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:21.994052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:22.007878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.008838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.009304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.009798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.010770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cb571f4363c34886b8763cab9310459f) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:22.011270 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.012892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.013070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d32d9-1a5f-4dd5-ab8b-cd478d75f381 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.019507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 340/684] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/role_assignments?user.id=cb571f4363c34886b8763cab9310459f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:22.061769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.062474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.062750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.063017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.073411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.073411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:22.094850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:22.096061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:22.097246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:22.099334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 345/685] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:22.105552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.241702 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a27e7d-9096-4f78-85cd-7898bf04bdbb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.289917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.290805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.291170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.291531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.292367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:22.292764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.294324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.313388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 341/686] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:22.319576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.341694 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.345883 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:22.442139 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470a5971-3a97-49e5-84e7-d8bb629c3a6c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.477266 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dae820b-9c28-4f68-a91e-6d4615d36689 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.482511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.483376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.483796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.484208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.502360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:22.523201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.525272 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.525763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.526256 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.541445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:22.548842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:22.549764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:22.550730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:22.554062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 342/687] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:22.558431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.567751 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.569280 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.634377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:22.672944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc732ac-faa9-4061-80e9-93327740e10c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.714125 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.715498 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.715970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.716444 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.717295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:22.718212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.719796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.730427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 343/688] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:22.736053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.844788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d0ef67-f11c-4130-ac30-b398121d3b12 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:22.878382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:22.883005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:22.883864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:22.885368 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:22.885849 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:22.897148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 346/689] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:22.898790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:22.916064 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:22.918514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:22.935069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:22.936040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:22.936912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:22.936912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:22.936912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:22.939580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 344/690] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:22.946470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.006450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.115303 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05adc3-cf64-4a60-bbe8-e032f3c197d6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.154535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b754f8-8e08-48d2-aa31-4c046a2e1a9f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.180396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.181141 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.181419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.181795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.193052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.194780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.199165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.199986 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.200439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.200909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.201727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:23.202246 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.203787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c0fee-1903-4071-840c-9d567684afd0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-ac2d8cc7-34fe-4478-bbf5-c59750550e8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:23.210470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:23.211484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:23.212358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:23.212358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:23.212358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:23.212358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 347/691] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:23.213193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a6f3-eb06-4653-819d-dd3e30310ee8 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.213477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a6f3-eb06-4653-819d-dd3e30310ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.213743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083a6f3-eb06-4653-819d-dd3e30310ee8 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.219369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 348/692] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:23.219369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a6f3-eb06-4653-819d-dd3e30310ee8 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.228569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 345/693] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:23.230310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.230796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.231263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.250358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:23.251891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:23.283252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.330160 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 346/694] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:23.330474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.331684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.333129 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083a6f3-eb06-4653-819d-dd3e30310ee8 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.369492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.370451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.370902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.371377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.372314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:23.372837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.374437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.386692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 349/695] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:23.397643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.528296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b228c69-103b-4119-b173-9b3808315ecd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.545502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e847a-6e00-4493-9ac1-f2b72724c3a5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.598578 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.598804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.599219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.599494 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.599762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.600067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.600473 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.600877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.606733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.609424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.619060 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.620707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:23.621167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:23.622214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:23.623271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 350/696] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:23.628380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.648127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:23.649197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:23.650500 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:23.651660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:23.654046 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 347/697] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:23.660599 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.712523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.736758 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7729f-7d4b-4ff1-9d83-3815f8524deb None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.772128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.772745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.773017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.773283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.773972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:23.774315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.775801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.788723 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 351/698] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:23.789552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.817574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b1399-3c48-4d46-9a72-568fd5bcddc1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.860573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.861185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.861460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.861723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.862537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:23.862845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.864478 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.872514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.884495 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 348/699] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:23.885407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.896208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc4c0d-49b2-4806-bc7c-a3ccba90746a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:23.931746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:23.932391 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:23.932646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:23.932915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:23.945310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:23.946842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:23.963088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:23.965380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:23.967462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:23.967462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:23.967462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:23.972184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 352/700] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:23.972184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:23.978673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:23.998733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50fb08-a337-407b-9fbc-a82db3da10a9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.038908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.039532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.039848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.040281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.041151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:24.041460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.043502 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.058030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 349/701] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:24.058030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.114167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66775285-54a9-4ba2-bb5e-38e84642f83f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.145914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.153554 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.154223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.154499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.154764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.190285 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964f1649-114d-4adf-be14-4be521c87174 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.227422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.228048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.228321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.228585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.229322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:24.229622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.229735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.231167 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c548c1-57b3-4875-95d5-dccec54a76ec None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.231292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.250157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 350/702] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:24.251714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634efb-f657-451f-82b2-ff68d4be4dfc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.251996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634efb-f657-451f-82b2-ff68d4be4dfc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.252361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29634efb-f657-451f-82b2-ff68d4be4dfc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.258394 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 351/703] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:24.260411 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.260830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.261107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.283387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:24.284670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:24.288719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.358940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 353/704] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 14 01:18:24.360049 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.374079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 352/705] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:24.375313 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.407504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.539226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f8c661-7b37-4843-a2a8-d280821af6c5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.584299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa11ac3-09cf-4315-a0ed-e30fdc26bcd8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.603735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.604370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.604655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.604911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.605571 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.607287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.623453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.624092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.624363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.624627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.634838 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.637291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.639229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-65d2cd55-79c8-4f82-a406-23765e24cbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:24.656906 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:24.657931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:24.658863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:24.658863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:24.658863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:24.659034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 354/706] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:18:24 2022] POST /identity/v3/users => generated 254 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:24.663295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 353/707] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:24.669062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.677153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960afd-0331-4e33-b177-ca11a39aff52 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.677489 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960afd-0331-4e33-b177-ca11a39aff52 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.677714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9960afd-0331-4e33-b177-ca11a39aff52 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.681681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 354/708] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:24.687786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.688109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.688331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.721192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:24.723089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:24.802824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.804590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 355/709] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:18:24.806521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.822700 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d839f79-1a80-48fa-80db-bf630236f029 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.873551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.874611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.875077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.875528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.876372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:24.876867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.884061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.904079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 355/710] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:24.905378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:24.938464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb989ba-ed5d-4518-bcd1-b425dea815cf None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:24.963791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:24.977160 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:24.977804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:24.978108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:24.978375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] PATH_INFO: `/v3/services/network` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:24.978966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:24.980342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:24.992237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:24.993346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:24.994809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 356/711] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/services/network => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:25.000684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.027833 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d814240c-c6e3-4b16-b8e2-c03a1feb888b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.072646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.073692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.074222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.074690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.086685 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.088277 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:25.103229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:25.104195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:25.105399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 356/712] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:25.115091 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.123836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d98a6c7-b161-433a-999e-b86d94a14d64 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.126291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:25.182439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.183220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.183593 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.183954 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.184687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] RBAC: Adding query filter params (name=network) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.185085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.186701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.192035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 357/713] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:25.192960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.248451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c7b399-0bea-4d3a-ad86-1ef9b2207fda None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.253507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:25.294063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.294787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.295123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.295457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.296733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.297100 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.298913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.308734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0de2b7-43f1-4cd9-816b-dc3594c4266b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.319013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 357/714] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:25.319013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.350421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.351410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.351921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.352387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.353223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.354814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:25.364154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:25.365148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:25.366389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:25.368845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 358/715] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:25.369903 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.394326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:25.438036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f96263-d516-436d-b5e4-1a88dbb3c72a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.475438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.476055 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.476329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.476595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.477473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] RBAC: Adding query filter params (type=network) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.477773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.479848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.484694 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d290b-a1a8-4b52-8112-f52018e4b8b0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.490023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 358/716] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:25.490023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.516901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:25.562517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.565078 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.565686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.566123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.567015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=03da2d917c354a389dda2242ac5d4297) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.567493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.569111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.576974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 359/717] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/role_assignments?user.id=03da2d917c354a389dda2242ac5d4297&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:25.668537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f74452-7149-4541-b9c0-278ec1d1046f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.710890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:25.718761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.719435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.719709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.719975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.720637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.720941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.722745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9bfef-2b78-485f-b860-8cf396139737 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.728535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 359/718] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:25.734411 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:25.858393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cf55be-b9ab-4fa7-872a-ef570e9cb7c5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:25.894525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:25.895389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:25.895801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:25.896206 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:25.896973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:25.897435 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:25.898964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e297022-cb3d-4a66-a5f9-5390c2bb1b7c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:25.905492 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 360/719] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:25 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:26.474729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e64-4260-49fd-8b23-7ddd72cbe957 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:26.475285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e64-4260-49fd-8b23-7ddd72cbe957 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:26.475285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3315e64-4260-49fd-8b23-7ddd72cbe957 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:26.476875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 360/720] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:26.494785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:26.495706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:26.496193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:26.518355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:26.519883 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:26.565111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:26.608776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 361/721] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:27.039598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3315e64-4260-49fd-8b23-7ddd72cbe957 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:27.162098 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3315e64-4260-49fd-8b23-7ddd72cbe957 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:27.209432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:27.210459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.210963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.211456 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.229665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:27.234890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:27.235818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:27.236671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:27.239581 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 361/722] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:27.246006 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:27.424754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c62a7-206e-45c1-930c-46c1a8aa742f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:27.468095 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:27.468709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.468987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.469255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.469923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:27.470245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:27.471620 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e81150-ca11-4139-964f-f7ea4416954c None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:27.485748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 362/723] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:27.491191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:27.507268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae36b1-0c18-4ccf-b12c-f2f659e2e27a None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.507746 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae36b1-0c18-4ccf-b12c-f2f659e2e27a None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.507824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae36b1-0c18-4ccf-b12c-f2f659e2e27a None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.509404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 363/724] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:27.514566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.514846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.515122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.543009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:27.544208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:27.606372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:27.610359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:27.651714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a9be2-338e-4afb-89b2-06c60ad22553 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:27.687012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:27.687563 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.687836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.688106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.700115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:27.701769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:27.717469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:27.718621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:27.719507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:27.719507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:27.719507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:27.719507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 362/725] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:27.724151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 364/726] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 14 01:18:27.728122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:27.861685 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cae8b2c-6982-4fa3-a120-be6a6503aa27 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:27.868807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:27.896333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:27.896949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:27.897269 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:27.897541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:27.898205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:27.898502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:27.899873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:27.918704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 363/727] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:27.924443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.009272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.086645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdfdc91-a0d7-401f-9c9a-6670cd67d978 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.125680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.126606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.127052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.127464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.136028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.137779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:28.150482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:28.151537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:28.152481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:28.155112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 365/728] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:28.161089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.189675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe14c507-951a-4a84-a747-44c5450fa179 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.210482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.278146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.278714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.278990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.279255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.280845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6eefb2-d236-460c-bc3f-44d9824cd1aa None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.316877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.318001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.318475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.318918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.319832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:28.320329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.321966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.326664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:28.331690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:28.332636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:28.333507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 364/729] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:28.342969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.346429 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 366/730] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:28.352554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.382636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.446962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.465739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae35571-d1bc-47ec-86c2-ee206664fc76 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.466646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea0d7b7-1fe8-46ae-b842-c2d6037a0f8e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.501129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.501489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.501941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.502193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.502282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.502355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.502424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.502699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.503385 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:28.503708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.505087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2ab18b-d124-447e-89ac-9569ed5b30dc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.512966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.513417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 365/731] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:28.514731 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95aec-70db-4937-945b-0ebccc28d662 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.515016 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95aec-70db-4937-945b-0ebccc28d662 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.515282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c95aec-70db-4937-945b-0ebccc28d662 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.515611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.521057 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 366/732] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:28.522261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e2b1-de6e-4489-927a-1386d42a4d2d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.522519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e2b1-de6e-4489-927a-1386d42a4d2d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.522799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493e2b1-de6e-4489-927a-1386d42a4d2d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.524167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 367/733] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:28.528937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e2b1-de6e-4489-927a-1386d42a4d2d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-4968db9b-9c5e-4acb-9308-0526b3cf16ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:28.533681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:28.534839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 367/734] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f233b-987a-49fd-8d4b-2143a45837af None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f233b-987a-49fd-8d4b-2143a45837af None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.541034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418f233b-987a-49fd-8d4b-2143a45837af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.555856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-418f233b-987a-49fd-8d4b-2143a45837af None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:28.557129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-418f233b-987a-49fd-8d4b-2143a45837af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:28.629359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.633961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 368/735] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:28.635317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926080-6b33-4ab4-b1c8-0062476d3ee7 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.635597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926080-6b33-4ab4-b1c8-0062476d3ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.635862 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c926080-6b33-4ab4-b1c8-0062476d3ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.654977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-5c926080-6b33-4ab4-b1c8-0062476d3ee7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:28.656019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493e2b1-de6e-4489-927a-1386d42a4d2d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.656141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5c926080-6b33-4ab4-b1c8-0062476d3ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:28.696352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.696992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.697266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.697541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.711680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.713693 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.726773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:28.728382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:28.729489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:28.730438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:28.730438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:28.730438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:28.730438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 368/736] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:28.730952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.741384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 369/737] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:28.743378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1497e-a052-4fa9-ac57-9476a10a5c29 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.743814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1497e-a052-4fa9-ac57-9476a10a5c29 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.744234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1497e-a052-4fa9-ac57-9476a10a5c29 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.750415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 370/738] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:28.751708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1497e-a052-4fa9-ac57-9476a10a5c29 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.844681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a2c69e-4a15-439d-bb33-848eee801807 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.865100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1497e-a052-4fa9-ac57-9476a10a5c29 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:28.881850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.882670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.883041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.883397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.884135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:28.884530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.886119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceff35f-546c-4d66-97c2-dd65ffd73dc7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.897921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.902858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 369/739] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:28.904211 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.904487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.904754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.911983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:28.913818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:28.914328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:28.914786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:28.915641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:28.916127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:28.918281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:28.923259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:28.924443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:28.933131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 371/740] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:28.934620 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:28.991477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:28.995468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 370/741] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 14 01:18:28.996521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.045959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088fcf81-46d0-4205-9189-88896f9996c2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.082371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.083091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.083459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.083692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.084430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.085809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.104560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 372/742] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:28 2022] POST /identity/v3/services => generated 240 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:29.105305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.126781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d68c68-26d2-4066-bd7d-8dd76f6262b7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.168681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.193717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.195333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.195623 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.195896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.207221 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.208892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:29.224252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:29.225262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:29.226622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:29.230548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 371/743] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:29.231517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.237441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef6d57-bacd-4cba-8359-7aaca0fff22a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.254308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.280274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.280926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.281200 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.281468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.290193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.291654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:29.302901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:29.303848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 373/744] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:29.304762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.354893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6946511-24b4-4f2e-9f1f-89518e4636e7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.389334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.390278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.390757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.391026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.391658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.393050 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.408094 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fa797-346e-4d1d-a0fc-6ee6597d4c55 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.410606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 372/745] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:29 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:29.411254 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.461480 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.462402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.462683 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.462957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.464680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:29.465021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.466520 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.478522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.484047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 374/746] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:29.485440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.534720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.561175 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed1c45-4145-4e8d-9d11-56543faff943 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.613556 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.614329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.614602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.614852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.626612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:29.631955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:29.632912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:29.633764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 373/747] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:29.634517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.636240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9111f-eb4b-438d-bd0e-0792dfa6f4e3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.685755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.687113 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.687786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.688226 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.689719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:29.690664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.693532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.715808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 375/748] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:29.717153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.770720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b2b716-65e3-408b-9585-d1710648a051 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.782660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.817696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.818428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:29.819773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.820528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.820945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.824258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=6f6158bf6948476a9244d0c28e1c8404, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.826680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.831614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba171c1-29e6-44cf-82b1-1424c65a23cf None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:29.850341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 374/749] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:29 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:29.851200 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:29.897957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:29.898537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:29.898808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:29.899073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:29.899758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:29.900115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:29.901514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:29.919697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 376/750] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:29.925951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.026863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ed663c-640f-46ac-8789-820d633a6c94 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.091614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.092247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.092529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.092799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.136087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.142055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2206488-d2d0-4549-8106-ecbb74c9681b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.179702 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.180047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.180381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.180654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.182947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.184701 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.192535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.194044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:30.210104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:30.211140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:30.212019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:30.212019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:30.212019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:30.212019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 377/751] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:30.217210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.220114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.308912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 375/752] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:29 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:30.314631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.322542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f2035a-7024-4abb-8a5e-8f5453a79122 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.358154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.358870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.359220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.359567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.360291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:30.360664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.362157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.374907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 378/753] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:30.380869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.393257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.440795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8de62b-52ea-451f-b664-c6615d81813e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.467580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.482605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.484501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.484965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.485420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.498063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.499999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:30.518133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:30.519225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:30.520101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:30.520101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:30.520101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:30.520281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c732ed29-9689-4209-959d-5b247d2b8347 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.523079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 376/754] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:30.529760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.555669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.556464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.558488 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.559097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] PATH_INFO: `/v3/users/swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.575316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.576796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:30.600786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:30.601767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:30.602738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:30.613926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 379/755] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:30.623796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.724749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de10dd-f184-4a6d-b9bf-4906d3441b8a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.776661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18967-a068-4749-97da-6f5bd767a0ed None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:30.779436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.813692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:30.822575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.823614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.824192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.824643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.825540 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:30.826021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.827648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.828218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:30.829094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:30.829465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:30.829825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:30.834690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:30.835116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:30.836719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:30.849332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 380/756] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:30.854631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 377/757] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:30.855755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:30.861107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.045443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b8309-3317-4963-abf7-cb2f86e182b7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.062770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793eaa9e-2066-4142-b379-f4106099d32d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.123199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.123837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.124111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.124373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.130499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.131114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.131386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.131650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.140806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.142286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:31.155679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:31.156636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:31.157547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:31.158000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 378/758] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:31.163862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.207402 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:31.246170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:31.274628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.274969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a783a81-133f-469e-a0c0-7488d05c0944 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.276666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.318125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.318125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.318125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.318125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.318534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:31.318534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.318534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.329121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 379/759] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:31.334797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.397944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 381/760] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:31.407013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.438340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:31.443777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b61a10-a4e2-4e48-b8b3-56172cc24409 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.481372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.482608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.483239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.483719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.484743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cf3b078d5a134266a19c889b477fb60f) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:31.485253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.486919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.493434 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 380/761] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:31 2022] GET /identity/v3/role_assignments?user.id=cf3b078d5a134266a19c889b477fb60f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:31.528773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:31.561438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b678990f-675e-4a52-ac9e-e2e778377c78 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.609841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.610552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.610828 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.611100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.622984 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.624518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-044549ce-e2c3-411c-859a-d87dd71f08c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:31.647317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:31.648334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:31.649205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:31.649205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:31.649205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:31.655949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 382/762] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:31.662079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.761432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862ba8-4fca-4ac3-bf5e-03cbd8ccb7f8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.761783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862ba8-4fca-4ac3-bf5e-03cbd8ccb7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.761974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c862ba8-4fca-4ac3-bf5e-03cbd8ccb7f8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.763628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 383/763] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:31.769127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444710-dcc7-4dc2-8460-75ae59f1f622 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.769405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444710-dcc7-4dc2-8460-75ae59f1f622 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.769675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444710-dcc7-4dc2-8460-75ae59f1f622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.793133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-99444710-dcc7-4dc2-8460-75ae59f1f622 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:31.794428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-99444710-dcc7-4dc2-8460-75ae59f1f622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:31.844018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfab5c2-43f5-4d29-a2ea-7908165a60f1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:31.874173 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 384/764] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:31.875384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d54d1-e77c-42ea-b1c6-05c81fb1e98f None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.875669 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d54d1-e77c-42ea-b1c6-05c81fb1e98f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.875944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d54d1-e77c-42ea-b1c6-05c81fb1e98f None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.883690 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 385/765] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:31.884883 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.885165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.885437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.898853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:31.899478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:31.899760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:31.900028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:31.900688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:31.900992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:31.902420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:31.918937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:31.920597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:31.923666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 381/766] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:31.929839 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:31.977443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.038260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.071727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 386/767] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:32.088309 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352f0720-b44b-423d-ac56-fec2e801720d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.130008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.130628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.130900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.131163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.140508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.141950 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:32.158511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:32.159453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:32.160411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 382/768] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:32.165689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.204829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.323477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.345179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c6be2-6490-44bf-bd79-f1f879ecb0ce None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.362785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c82711-0153-4850-bc4d-408f937d15ad None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.386717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.387259 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.387525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.387790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.388487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:32.388786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.390271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.410200 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 387/769] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:32.411242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.425721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.426528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.426889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.427154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] PATH_INFO: `/v3/services/network` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.427746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.429460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:32.442645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:32.443726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:32.444832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 383/770] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services/network => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:32.445828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.484148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.535813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8182e17-9d36-4372-ac53-3964c2803fbc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.569191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a0f2e5-aa37-4056-8462-1df87e344212 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.571328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.572899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.573178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.573442 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.574075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.575467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:32.585891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:32.586926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:32.588371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 388/771] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:32.589187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.615283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.616456 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.617364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.618121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.619326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=03da2d917c354a389dda2242ac5d4297) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:32.620186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.622064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024ccf56-baae-4ceb-bfa1-29506dce2532 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.629711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 384/772] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/role_assignments?user.id=03da2d917c354a389dda2242ac5d4297&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:32.631628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453f0-0333-4a65-afbd-7fc6e3a20e8d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.632431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453f0-0333-4a65-afbd-7fc6e3a20e8d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.633127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae453f0-0333-4a65-afbd-7fc6e3a20e8d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.635256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 385/773] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:32.637081 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae453f0-0333-4a65-afbd-7fc6e3a20e8d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.653695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.704320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37409914-4bd3-4d05-950b-92561c75e356 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.723907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:32.747859 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.748513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.748762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.749027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.749795 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] RBAC: Adding query filter params (name=network) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:32.750140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.751743 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c999e4-31a1-46db-8800-6ffe03b750dd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.757324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 389/774] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:32.763638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.763638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.763638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.780842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:32.782109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:32.792308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae453f0-0333-4a65-afbd-7fc6e3a20e8d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:32.832037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:32.833065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:32.833549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:32.834070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:32.834920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:32.835449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:32.837728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:32.845278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 386/775] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:32.847073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.870020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 390/776] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:32.870020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:32.987367 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.025411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751cb4d-b5ac-4f28-b61b-46c1e66bab7b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.053871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f595e0-7f50-4133-91d7-020f7637dccf None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.061873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.062778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.063246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.063706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.064897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] RBAC: Adding query filter params (type=network) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:33.065526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.067177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.079853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.093088 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 387/777] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.094559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.119957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.120576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.120849 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.121164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.121807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:33.122151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.125124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.134787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 391/778] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:32 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.135636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.228861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9cf4f8-6db1-4581-a09a-ba59e0267060 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.265878 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a538c70-4ebf-4ec5-a2c5-4d848bad14be None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.266802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.267613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.268042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.268452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.269357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] RBAC: Adding query filter params (interface=public, service_id=cc1e06ce4bf746608748857fcb749460, region_id=RegionOne) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:33.269898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.271445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.279855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 388/779] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/endpoints?service_id=cc1e06ce4bf746608748857fcb749460&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.281103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.285043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.308685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.309327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.309778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.310210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.326141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:33.332220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:33.333332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:33.334288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 392/780] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:33.339868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.384404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.404336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ce01-8826-4004-b4b2-a5c8f8d665fe None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.445999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.447051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.447551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.447997 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.448888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] RBAC: Adding query filter params (interface=public, service_id=cc98f45853d44c02b4531376e1618aa3, region_id=RegionOne) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:33.449705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.451714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.458820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 389/781] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/endpoints?service_id=cc98f45853d44c02b4531376e1618aa3&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.460008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.473653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45358cf-ddfa-48a8-84f4-142865e25072 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.515744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.516728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.517183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.517628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.518562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.520231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.526504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 393/782] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/services => generated 772 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.527741 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.542813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.620485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6de5239-b5cc-45ae-866d-d5f2e5c7456b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.664634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.665727 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.667114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.668281 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.670544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:33.671107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.675430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.693738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 390/783] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.700109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:33.745300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.752074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a658f6f-763b-46af-ae5a-5add8c7fb2a2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.804454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.805057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.805366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.806140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.806774 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.808861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.816882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 394/784] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/services => generated 772 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:33.833451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9edd5-9b08-498b-904d-48e0e6122350 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:33.871958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:33.883436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:33.884126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:33.884396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:33.884710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:33.898612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:33.900128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-7be4c5b8-aaba-4277-bd83-ec9a6e761586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:33.918035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:33.919072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:33.919993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:33.919993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:33.919993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:33.919993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:33.926369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.008168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11cc6-0ecc-47cb-b6ba-7ad3e8dcd1cc None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.008846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11cc6-0ecc-47cb-b6ba-7ad3e8dcd1cc None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.009337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d11cc6-0ecc-47cb-b6ba-7ad3e8dcd1cc None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.011382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 392/786] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:34.018270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.018965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.019458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.039847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:34.041640 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:34.100131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacbad59-aa20-46db-a958-cefe1005ac7e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.127961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 393/787] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:34.140072 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:34.140692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.141015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.141287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.141935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:34.142306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:34.143773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:34.156441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 395/788] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:34.162112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.271263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a09ae70-a2cc-4cc7-ab2a-f7be1cf2c489 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.289464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:34.319303 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:34.319942 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.320215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.320478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.327945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:34.329475 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:34.341525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:34.342518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:34.343472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 394/789] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:18:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:34.351117 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.377063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:34.455178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b0a486-6648-4c7e-8738-e5e38cc254fd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.489562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:34.490343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.490625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.490897 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.491628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:34.491928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:34.493378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:34.511975 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 396/790] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:18:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:34.518249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.579421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.647356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c3580-d5da-47d1-8afa-0d3cb5e28a6a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.727382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:34.728986 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:34.729713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.730006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.730314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.765557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025b6d37-e2bf-4ab5-8bbc-c808ae59decc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.799397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:34.836043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:34.836603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:34.836876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:34.837142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:34.858753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:34.860692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:34.861015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:34.867843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:34.868799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:34.869649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:34.870872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 397/791] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:34.877273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:34.989839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2022002-4a91-4148-86fc-17f746cba047 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:34.996870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 395/792] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:35.003763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.037134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.037766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.039484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.039760 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.040476 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:35.040774 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.042529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.060708 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 398/793] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:35.066729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.082680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:35.134401 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:35.175216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a4552-1a77-4fda-b940-e135ecbe6c9d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.223970 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.224614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.224890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.225158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.229401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f5543-014f-47de-97d0-068e0ea083ec None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.237266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.239242 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-412746f1-fb67-4fbc-8274-85d927ce3168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:35.257809 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:35.258783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:35.259676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:35.259676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:35.259676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:35.262179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 396/794] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:35.263490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41e620-630f-4ed2-af4d-54139b84d617 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.263929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41e620-630f-4ed2-af4d-54139b84d617 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.264326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa41e620-630f-4ed2-af4d-54139b84d617 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.270084 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 397/795] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:35.271418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41e620-630f-4ed2-af4d-54139b84d617 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.275557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.277102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.277389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.277663 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.290422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.292643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-69f25130-68dd-46e2-80f7-5c3dba3d11e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:35.311658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:35.312595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:35.313491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:35.313491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:35.313491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:35.313491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 399/796] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:35.314846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.315155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.315430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.345019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:35.346567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:35.405625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:35.427170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 400/797] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:35.428069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.523002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa41e620-630f-4ed2-af4d-54139b84d617 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.563361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad2eba-f4bb-47cf-9324-f8fa9c082d3d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.574172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:35.593310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.595023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.595641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.597235 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.598194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:35.598823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.601259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.624733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 398/798] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:35.632388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.638560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.639410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.641538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.642028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.646836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:35.647346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.648939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.670119 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 401/799] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:35.671734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.723849 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:35.782077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af32020-ad55-458d-81f7-3f15026b4ddf None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.822824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.823461 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.823737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.824007 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.834220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.836327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.844767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edd0578-1a78-4a03-bb35-595a6ae76f5f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:35.850839 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:35.851805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:35.852852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:35.857274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 402/800] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:35.858332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.892530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:35.893718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:35.894032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:35.894524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:35.902713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:35.904221 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:35.918156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:35.919144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:35.920096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:35.925152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 399/801] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:35.926176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:35.982372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.001198 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6168b98f-ece2-4fe6-ad7e-ddcb993d29ca None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.005307 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.047684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.048319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.048589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.048853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.052225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a57e9-f826-454c-903b-8c218069fdda None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.059757 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:36.065390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:36.066390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 403/802] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:36.072554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.092695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.093433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.093553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.093822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.094600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.094899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.096315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c24593c-05f5-4563-b6aa-09a2f5ce6108 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.113642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 400/803] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.114889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf468b-896e-45ca-b66b-a69da83b7995 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.115166 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf468b-896e-45ca-b66b-a69da83b7995 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.115432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf468b-896e-45ca-b66b-a69da83b7995 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.120792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 401/804] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:36.121573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf468b-896e-45ca-b66b-a69da83b7995 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.178996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ba304-f87b-4d20-bc76-8c75cd9b2adb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.213332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.213939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.214242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.214507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.215199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.215499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.217018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.226564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.233880 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 404/805] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.233880 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.237283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf468b-896e-45ca-b66b-a69da83b7995 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.256087 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.274402 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.275006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.275270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.275535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.276184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.276617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.277848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8544a03b-37c1-45b2-b7dc-9046c969dd90 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.290713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 402/806] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.290713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.290713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.290713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.306855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:36.308222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:36.339224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b780c719-6b22-412d-af83-18e7dac8593d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.374734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.375397 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.375666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.375934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.384034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 403/807] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:36.384807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.395194 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.397768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:36.418900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:36.419949 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:36.420892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:36.420892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:36.420892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.423697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 405/808] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:36.424650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.490560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.494792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673a7da8-f552-4654-8951-2fc96febeef6 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.518078 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.536643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.537344 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.537616 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.537879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.538629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.538935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.540566 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.542820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f52c6d-2c44-4d6a-a952-e9d6b4f74fd9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.549953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 404/809] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.551132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.591689 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.592813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.593311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.594032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.610344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.612900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-82af96b9-9c6f-43fb-bec2-ca8de859d25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:36.635001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:36.636030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:36.637339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:36.637339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:36.637339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.637339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 406/810] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:36.637562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe653541-4150-42fd-86c4-c23cab470996 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.637821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe653541-4150-42fd-86c4-c23cab470996 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.638036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe653541-4150-42fd-86c4-c23cab470996 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.649118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 407/811] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:36.652150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.652449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.652672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.676632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:36.678011 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:36.684288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed1a2b3-9b5e-43d7-abfd-20bb48c0c8ba None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.721219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.721870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.722179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.722458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.723219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.723495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.724931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.739662 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 405/812] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.741126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.753471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 408/813] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:36.754357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.766574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.820073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:36.864465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f588fb-fddc-45c4-9c9e-0791641a0b8c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.872301 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698abef-9110-4f2e-aa00-30eb6a5c323f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:36.910817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.911431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.911696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.911964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.912662 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:36.912961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.914129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:36.914415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.914766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:36.915039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:36.915307 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] PATH_INFO: `/v3/services/network` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:36.916061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:36.917900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:36.927453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 406/814] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:36.928338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:36.928590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:36.929559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:36.930408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 409/815] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:36.936514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.028717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.043558 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4e3b0b-020d-45c0-97aa-c9180f19ba01 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.056606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9134bcd-56dd-42fc-aedd-47a24a679724 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.068148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.080913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.081559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.081941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.082141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.082784 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] RBAC: Adding query filter params (name=network) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:37.083083 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.084463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.089140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 410/816] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:37.089927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.107802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.108852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.109293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.109744 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.117149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.118854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:37.130113 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:37.131031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:37.131911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 407/817] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:37.132937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.200487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c72e38-74ea-45cc-b096-7857da23d161 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.234572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.235309 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.235470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.235738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.239990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fa293-93e8-40da-ae60-83be85f87009 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.243398 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.244837 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:37.255993 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:37.256926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 411/818] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:37.257899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.281603 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.282219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.282497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.282797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.283370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.284780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.288535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:37.296726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:37.297766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:37.298654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 408/819] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:37.299460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.315182 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.371853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16bd0fd-a0ab-4329-963c-5a1c0ffd0b36 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.406978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79feeecd-bae2-4e95-84b7-786a8d91757a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.409467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.410191 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.410467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.410732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.411364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] RBAC: Adding query filter params (type=network) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:37.411661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.413039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.418404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 412/820] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:37.423089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.448360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.448972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.449253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.449524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.450214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:37.450526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.451973 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.469264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 409/821] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:37.469264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.530467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.536855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d355d-9e95-4678-b89f-2459d86873ca None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.547355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.572146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.572752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.573057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.573380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.574148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:37.574535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.575899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.576591 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c6c3-8df1-45c7-97fc-4292aefdb8f4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.613587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.614265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.614536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.614799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.615438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:37.615736 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.617138 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.627650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 410/822] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:37.627650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.712384 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 413/823] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:37.713185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.740973 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e026fe1-7c29-4a9e-a469-0bd6f2cbd84a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.768182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.781731 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.782415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.783268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.783549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.784599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.786015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.818112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 411/824] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:18:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:37.818112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.827874 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2616aa77-6e7f-4b58-9170-28c33eaea137 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:37.874401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:37.875261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:37.875655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:37.875998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cf3b078d5a134266a19c889b477fb60f/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:37.894169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=cf3b078d5a134266a19c889b477fb60f, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:37.896057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:37.918102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 414/825] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:37 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cf3b078d5a134266a19c889b477fb60f/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:37.919215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:37.953686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:37.972853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36650d6c-91fc-4a1b-838c-25671a7c1e53 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.057329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.058019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.058145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.058403 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.059147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:38.059446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.060840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.072943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 412/826] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:38.079927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:38.139840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565e638-3b4c-450a-8ca2-0d33acfe0c0a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.147195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:38.179800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.180042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.180288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.180505 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.191631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.193096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:38.208948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:38.209942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:38.210892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:38.210892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:38.210892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:38.210892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 415/827] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:38.216132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:38.228303 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10ee4c-157e-4184-a7c9-238bf2c87051 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.280133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.281422 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.282182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.284409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.285613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.288247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.290904 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:38.318570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 413/828] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:18:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:38.345273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1d85-beb4-4faf-b2d4-16e5aafc1d93 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.392176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.392806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.393073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.393346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.394133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:38.394497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.395983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.413813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 416/829] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:38.416114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:38.532264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:38.536518 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63c1583-1916-4e83-a7a1-01de68c63cbd None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.578430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.579521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.580020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.580487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.593734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.595651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:38.608315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:38.609321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:38.610286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:38.612709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 414/830] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:38.618349 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:38.727249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd03bf1-b424-476e-b76c-0c22c27e195b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.754769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:38.768142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.768805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.769079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.769405 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.770140 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:38.770445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.771882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710522db-4818-4fa2-b7c0-54ffea050909 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.784853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 417/831] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:38.790895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:38.838504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afcd7-760c-483a-8c8f-5ccfe20e8b35 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.839063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afcd7-760c-483a-8c8f-5ccfe20e8b35 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.839063 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1afcd7-760c-483a-8c8f-5ccfe20e8b35 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.841624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 418/832] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:38.845541 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.845866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.846110 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.864942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:38.866534 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:38.902933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413137a5-ccc2-4121-ab0e-bf7ee9a6cc3d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:38.936976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:38.937721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:38.938015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:38.938278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/03da2d917c354a389dda2242ac5d4297/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:38.940878 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=03da2d917c354a389dda2242ac5d4297, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:38.942356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:38.945636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 419/833] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:38.962482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 415/834] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:38 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/03da2d917c354a389dda2242ac5d4297/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:39.210351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:39.310830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99260121-9f6c-451f-b0f2-125eab669261 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:39.325359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:39.353371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:39.354420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:39.354890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:39.355346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:39.356141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:39.357693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:39.366953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:39.367875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:39.368734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:39.376510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 420/835] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 14 01:18:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:39.378607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:39.402865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:39.482658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e20666-f2b5-436b-894f-53ae735f99ac None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:39.516934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:39.517551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:39.517828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:39.518128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:39.518810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:39.519116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:39.520498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e5ac2-0172-4462-a290-fed3c51697e8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:39.525336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 416/836] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:39.531306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:39.644199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411b611-70d1-4ae6-94dc-c4684ef05733 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:39.680678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:39.681291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:39.681568 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:39.681835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:39.682499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:39.682799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:39.684174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbdcdcb-16df-4be5-b235-4f65e982b12b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:39.689544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 421/837] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:39 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:40.161162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6b39a-8652-4db5-b649-0d5545a47ff4 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.161503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6b39a-8652-4db5-b649-0d5545a47ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.161717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e6b39a-8652-4db5-b649-0d5545a47ff4 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.163388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 417/838] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:40.168962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef0727-766f-4c39-8874-4a70c9ebed33 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.169418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef0727-766f-4c39-8874-4a70c9ebed33 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.169494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ef0727-766f-4c39-8874-4a70c9ebed33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.188347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a0ef0727-766f-4c39-8874-4a70c9ebed33 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:40.189940 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a0ef0727-766f-4c39-8874-4a70c9ebed33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:40.227070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:40.261490 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 422/839] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:40.525651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6b39a-8652-4db5-b649-0d5545a47ff4 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:40.621119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:40.639637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e6b39a-8652-4db5-b649-0d5545a47ff4 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:40.674468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:40.675498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.675989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.676449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.687780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-dd190d32-4995-4328-9c9f-894a70f79d62 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:40.693519 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:40.694473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:40.695311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 418/840] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:18:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:40.695988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a021-c87a-47e4-9651-56856a68f3fc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.695988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a021-c87a-47e4-9651-56856a68f3fc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.695988 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc7a021-c87a-47e4-9651-56856a68f3fc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.697112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 423/841] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:40.702860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a021-c87a-47e4-9651-56856a68f3fc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:40.704482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.704755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.705020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.723747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:40.724937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:40.793134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 419/842] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:40 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:40.828081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc7a021-c87a-47e4-9651-56856a68f3fc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:40.863245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:40.863899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:40.864180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:40.864453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:40.865123 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:40.865422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:40.866845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:40.875532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 424/843] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:18:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:40.881458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:40.962451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:40.988928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bb1cdb-708c-4c63-bcfe-a6c924a7ffaa None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.024001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.024625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.024894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.025156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.035484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.036929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:41.051933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:41.052880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:41.053780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:41.053780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:41.053780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:41.053780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 420/844] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:41.056992 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.060617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.105916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.180853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082a41de-24d8-4905-a704-edc118fd6649 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.207451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.210400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fd5a8-dc00-4da6-abd9-b00360b13537 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.218118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.218732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.219003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.219268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.219901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:41.220193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.221594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b092b9-33a3-4fa8-9bab-f2692b8e4353 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.238945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 421/845] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:41.240301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92151df-c5f7-4cf9-ac62-6bc613faed8f None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.240583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92151df-c5f7-4cf9-ac62-6bc613faed8f None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.240850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92151df-c5f7-4cf9-ac62-6bc613faed8f None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.242343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 422/846] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:41.242903 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92151df-c5f7-4cf9-ac62-6bc613faed8f None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.261803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.262555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.262716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.262974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.263551 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.264910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-73629730-7c2f-41bd-b05a-91b959633c79 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:41.273670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:41.274686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:41.275565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 425/847] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:41.276347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.276347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.277121 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.293946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:41.295155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:41.358745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92151df-c5f7-4cf9-ac62-6bc613faed8f None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.366166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 426/848] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:41.366166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.391767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.401877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.402573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.402916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.403254 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.411302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.412925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:41.427079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:41.428043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:41.428975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:41.433441 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 423/849] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:41.434637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.488195 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a4d6d-6073-4a77-a469-26f9f2e0bc81 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.491053 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.526339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.527388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.527388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.527682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.528381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:41.528744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.530388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.539861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 427/850] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:41.540879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.568763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024fa6-c81f-453d-ae49-7132600d1c13 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.612246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.613096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.613779 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.614598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.615819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:41.616278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.618120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.639698 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 424/851] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:41.641240 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.649798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.659657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f9cc84-cece-43ab-bbb9-a8e4e39cd790 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.695434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.696044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.696319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.696589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.697225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:41.697532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.698927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.714021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 428/852] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:41.714021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.780237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ab0f3-9bfb-4a1f-a07b-8ed424661581 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.784574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.818677 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c879cd8-a2a8-4678-b92f-f34464df8333 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:41.829482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.829784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.830138 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.830418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.842756 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:41.847936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:41.854185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:41.855165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 425/853] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:41.855880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:41.871454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:41.872444 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:41.872918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:41.873372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:41.904450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:41.950776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:41.952600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58346162-d216-40ef-b45b-1f3bc05e5e2d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:41.972108 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516014f7-8caa-4463-8d9f-b01724c2f5de None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.016932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.017807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.018273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.018653 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.019381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:42.019608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.020990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102fde4-d33b-4352-aa0c-8cd05964e9b1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.031151 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 426/854] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:42.032517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eeb32-6347-4b23-96c4-f74573431a8d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.032786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eeb32-6347-4b23-96c4-f74573431a8d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.033054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64eeb32-6347-4b23-96c4-f74573431a8d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.034523 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 427/855] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:42.039598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eeb32-6347-4b23-96c4-f74573431a8d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.061844 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 429/856] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:42.063588 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.063863 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.064477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.075128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.090637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:42.091833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:42.140395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.155919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64eeb32-6347-4b23-96c4-f74573431a8d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.166027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 430/857] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:42.166027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.202371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.203102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.203439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.203772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.217553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.219214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:42.236977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:42.238042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:42.238945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:42.238945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:42.238945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:42.239116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 428/858] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:42.246622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.278036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94401a-a5b2-4e44-8a1d-963e1bf31819 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.316396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.317045 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.317314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.317619 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.329440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.331028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.333761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:42.348243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:42.349230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:42.350139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:42.350139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:42.350139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:42.354149 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 431/859] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:42.355570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.361620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376abe91-67c7-4e39-ac24-000593ff3113 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.392306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.399123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.400032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.400740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.401858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.402925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:42.403417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.405197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.423671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 429/860] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:42.425207 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.461145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e452fec4-b172-4039-a388-8ca8e737ced5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.495056 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.495659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.495943 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.496200 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.496872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:42.497164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.498544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.514041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 432/861] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:42.514041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.533099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44e736-7dfe-4750-90d2-e81b2c5ada0b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.569509 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.570153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.570424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.570689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.579583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.581157 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.584689 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:42.596340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:42.597423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 430/862] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:42.598397 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.622673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.630448 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c981e-b5b3-4d80-962f-9a062f4a2626 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.666647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.667283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.667554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.667816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.668572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.669939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.690109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 433/863] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:42 2022] POST /identity/v3/services => generated 269 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:42.690109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.709710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134d9bd-6953-4311-a32c-6dba80c11e43 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.762625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.763283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.763554 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.763819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.773045 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.775706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:42.791874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:42.792846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:42.793720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:42.801217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 431/864] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:42.803831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.824050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077766d1-9593-4363-87a2-dbfcbde21f22 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.863076 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.864073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.864073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.864282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.865026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:42.865396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.866857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.874451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.890431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 434/865] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:42.896051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:42.913108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:42.927585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7b63f-985e-4460-9c73-8edfcc6e8b79 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:42.969094 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:42.969860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:42.970208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:42.970798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:42.971855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:42.972160 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:42.974143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:42.989594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 432/866] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:42.995574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:43.022527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4364-f650-4852-a7d4-0690b1894a90 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.066400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.067114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.067450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.067783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.106991 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b15327-0754-4811-87e0-7a2adbe81852 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.122681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:43.143755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.145210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.146924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.147824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.149027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.149446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.150346 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cf3b078d5a134266a19c889b477fb60f) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:43.150796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.152356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.160532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:43.162099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 433/867] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/role_assignments?user.id=cf3b078d5a134266a19c889b477fb60f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:43.264896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 435/868] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:42 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:18:43.270802 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:43.383240 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc49546-ea02-4af1-9b63-3609e5376d1b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.417942 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.418611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.418882 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.419147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.430882 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.432337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:43.448329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:43.449314 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:43.450289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:43.450289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:43.450289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:43.450289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 434/869] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:43.456958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:43.559427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce1f942-8c02-45ed-b0f9-f64d22340160 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.574390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:43.596177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.596816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.597091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.597359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.598097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:43.598399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.599830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.611976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 436/870] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:43.617675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:43.651533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:43.737907 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77d394-93ba-40d1-b4c9-0d8c5afcecf1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.773773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.774430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.774698 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.774959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.781702 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.783230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9238c827-1fc4-412e-876c-b7a385171182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:43.794850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:43.795814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:43.796684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 435/871] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:43.803833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:43.906824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba203c5-5050-45f9-acd2-04433d39574e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:43.921780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9541-0a5f-41ff-a42a-62e64dbff219 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.922127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9541-0a5f-41ff-a42a-62e64dbff219 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.922358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d9541-0a5f-41ff-a42a-62e64dbff219 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.923968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 436/872] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:43.931196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.931474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.931740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.942440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:43.943390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:43.943857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:43.944316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:43.945196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:43.945717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:43.947447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:43.953225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:43.954600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:43.964523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 437/873] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:43.970151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:44.007040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:44.045734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 437/874] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:43 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:44.086646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14fad63-5bb5-4e48-9d50-e8fc78392fb1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:44.115836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:44.116497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:44.116769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:44.117035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:44.117813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=03da2d917c354a389dda2242ac5d4297) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:44.118149 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:44.119648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703f6dcb-1806-4181-a2e8-9431941684f7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:44.126034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 438/875] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:43 2022] GET /identity/v3/role_assignments?user.id=03da2d917c354a389dda2242ac5d4297&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:44.326797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:44.350426 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:44.441092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12d81a2-9297-4c4b-98d3-f10eb9f49eeb None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:44.475530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:44.476136 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:44.476406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:44.476670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:44.477319 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:44.478714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd89526-49ed-4f33-a0ae-1d6649f372c8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:44.495691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 438/876] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:44 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:44.985077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478cd-d68a-45f7-8ace-12b4dba14c71 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:44.985606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478cd-d68a-45f7-8ace-12b4dba14c71 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:44.985606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738478cd-d68a-45f7-8ace-12b4dba14c71 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:44.987364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 439/877] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:44.992705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fcc5-87d3-48ae-808d-9b3a6857ef49 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:44.993157 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fcc5-87d3-48ae-808d-9b3a6857ef49 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:44.993237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d3fcc5-87d3-48ae-808d-9b3a6857ef49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.011492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d7d3fcc5-87d3-48ae-808d-9b3a6857ef49 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:45.012684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d7d3fcc5-87d3-48ae-808d-9b3a6857ef49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:45.084714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 439/878] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:44 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:45.348701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738478cd-d68a-45f7-8ace-12b4dba14c71 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:45.461857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:45.467034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738478cd-d68a-45f7-8ace-12b4dba14c71 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:45.497751 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035f79-18aa-4dc0-826f-77762a4c53cf None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.498112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035f79-18aa-4dc0-826f-77762a4c53cf None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.498334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97035f79-18aa-4dc0-826f-77762a4c53cf None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.499913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 440/879] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:45.502600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:45.503287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.503554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.503812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.504391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:45.505172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.505550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.505751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:45.506036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:45.515832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:45.516970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:45.518483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 440/880] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 14 01:18:45 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:45.520166 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:45.523300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:45.528486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:45.529967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:45.610355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 441/881] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:45.628065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b6ae79-9c4f-4ac9-aa06-f15037b45bd1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:45.662567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:45.663408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.663826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.664262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.665031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:45.665472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:45.667018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:45.672321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 441/882] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:45 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:45.678334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:45.786667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03828f0-47bb-4a03-a059-377baee3842c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:45.822279 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:45.823145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.823563 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.824000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:45.824768 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:45.825209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:45.826743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:45.829005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:45.833162 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 442/883] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:45 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:45.839239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:45.868609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:45.908452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:45.955433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c290355-9629-4eaf-909b-2d9b598a1fb1 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:45.996591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:45.998300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:45.998720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:45.999443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.000616 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] RBAC: Adding query filter params (interface=public, service_id=298e3fef227b4ac08aaf230cc00ea386, region_id=RegionOne) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:46.001070 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.003256 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.017780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 442/884] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:45 2022] GET /identity/v3/endpoints?service_id=298e3fef227b4ac08aaf230cc00ea386&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:46.019399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.028803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6d2b55-e5e1-41f6-a8a6-e33675ec6856 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.075863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.076505 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.076774 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.077033 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.095562 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:46.102372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:46.103287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:46.104165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:46.106728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 443/885] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:18:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:46.112390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.155909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:46.169654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddac55c-3549-45fe-8229-6b57ba095fca None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.181546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:46.218873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.220032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.220504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.221281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.222446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.224895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.227025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d57d05-0b70-4383-8c18-9e56ebed2b48 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.241499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 443/886] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/services => generated 1268 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:18:46.269466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.270133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.270410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.270675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.271348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:46.271650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.273150 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c081a-23e6-48f6-b2aa-7d1165e7a399 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.283251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 444/887] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:46.289378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.406082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85c260e-b894-4f86-b248-fdad8824604a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.429172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2d014-3321-41ed-88a9-85a179367381 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.429835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2d014-3321-41ed-88a9-85a179367381 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.430328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d2d014-3321-41ed-88a9-85a179367381 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.432186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 445/888] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:46.438063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.438627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.439495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.443945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.444587 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.444939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.445215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.459326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.460778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.461636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:46.462975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:46.479240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:46.480220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:46.481101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:46.481101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:46.481101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:46.481101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 444/889] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:46.485424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.497193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:46.548686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 446/890] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:46.608558 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf9a6e3-7164-4718-a124-6019fccebbc4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.647790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.648825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.649298 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.649762 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.650813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:46.651311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.652876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120dae6a-31ee-46c6-a340-7cdb9c51d4d3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.670292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 445/891] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:46.672169 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.727958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:46.732282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ec50-27f2-487b-b72f-80b61f5c7903 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.732565 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ec50-27f2-487b-b72f-80b61f5c7903 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.732860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1ec50-27f2-487b-b72f-80b61f5c7903 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.734504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 446/892] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:46.740424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.740960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.741420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.759563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:46.761071 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:46.790369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e45ad7-56d1-4524-9056-4fb1f2cce306 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.822938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:46.831587 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 447/893] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:46 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:46.838178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.838258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.838258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.838432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.838918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.849006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.851095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:46.867657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:46.868619 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:46.869540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:46.874105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 447/894] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:46.874978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:46.940353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9601da-d58b-40a4-9639-f9a2d828c202 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:46.986562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:46.987645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:46.987926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:46.988590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:46.989445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:46.990855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:46.990980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854cc60-756a-4f5c-b446-d92482e2a00c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:47.002642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:47.004090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:47.005395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.006202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.006277 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 448/895] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:47.010317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.029202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.029852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.030149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.030415 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.031253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.031552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.033028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.056829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 448/896] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:47.057751 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.087162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.135048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f85c3a-b772-4419-8fbb-620d65473f33 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.164334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff6662-ff75-4a45-940f-d6de0ffce5e3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.169035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.169889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.170449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.170885 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.171783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.172269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.173977 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.183832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 449/897] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:47.183832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.198525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.199060 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.199333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.199599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.242732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.272327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.273805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.293923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5529b9b3-aff2-4042-835d-f81f7162746b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.320244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.333133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.334018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.334486 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.335131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.336028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.337581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:47.347215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:47.348306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:47.349430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.351169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 450/898] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:47.351932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.387765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 449/899] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:47.387765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.451318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.462116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f147432-6e9c-4be9-bbaa-06a7910be2a3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.483690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43597bc4-cf59-45b9-bfb7-3f7674f80d58 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.496833 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.497447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.497719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.497972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.498625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.498915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.500281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.514020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 451/900] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:47.514020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.530820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.531537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.531813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.532079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.532719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.533017 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.534520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.544779 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 450/901] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:47.546265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.575359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.647941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890c553-54cd-4034-af0b-a8c88354934d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.664633 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613dfd1f-50fe-4c40-ab6e-737922c2004e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.695662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.697152 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.697677 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.698126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.704680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.705226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.705501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.705833 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.706535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.706839 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.709286 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.711009 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.714089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.716021 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.718144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 451/902] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:47.723452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:47.734575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:47.735732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:47.736678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:47.736678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:47.736678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:47.738556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 452/903] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:47.744225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.844835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46daa1a6-ef43-43dc-a34e-724f6ab86f74 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.855921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c67dc3-7cd1-4213-8ce1-4b019c45c433 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:47.858672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:47.885475 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] RBAC: Adding query filter params (interface=public, service_id=1c22635d6b1b43b68b0856cbd2482012, region_id=RegionOne) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.894090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.895641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 452/904] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/endpoints?service_id=1c22635d6b1b43b68b0856cbd2482012&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:47.899758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:47.900622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:47.901103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.901219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:47.901759 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:47.902639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:47.903141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:47.906133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:47.927414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 453/905] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:47.928107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:47.977596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:48.025108 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12449df-6d1e-4e58-8eff-d8e44309de32 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:48.063219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ae29e6-7024-419f-a148-28bf2bdf33fa None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:48.064377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:48.064992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.065284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.065547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.066179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:48.068902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:48.082214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 453/906] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/services => generated 1268 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:18:48.125546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:48.126730 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.127218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.127841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.135917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:48.137837 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:48.150973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:48.151997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:48.152899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:48.155628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 454/907] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:48.162083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:48.273391 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b05e8d1-b089-478d-bb42-58105ce8e6c0 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:48.299915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:48.309783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:48.310424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.310692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.310952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.311634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:48.311928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:48.313337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7fc79-46a3-4388-a215-93fcb1eff6a6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:48.326605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 454/908] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:48.332835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:48.358867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:48.444686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283474d0-3691-43a4-b59c-8ce8fcceab12 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:48.482597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:48.483463 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.483990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.484416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.485300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] RBAC: Adding query filter params (role.id=16b8db405fe9477e8959d74aed6537e4, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cf3b078d5a134266a19c889b477fb60f) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:48.485743 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:48.487585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ddddc6-d31d-40af-b5e0-b31ba3cba565 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:48.493959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 455/909] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:48 2022] GET /identity/v3/role_assignments?user.id=cf3b078d5a134266a19c889b477fb60f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=16b8db405fe9477e8959d74aed6537e4 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:48.515663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b4d9-2422-41ec-9d57-396c6800d396 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.515946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b4d9-2422-41ec-9d57-396c6800d396 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.516227 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b67b4d9-2422-41ec-9d57-396c6800d396 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.517899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 455/910] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:48.524047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:48.524229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:48.524508 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:48.544111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:48.545320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:48.620894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 456/911] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:48 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:48.892445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b4d9-2422-41ec-9d57-396c6800d396 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:48.998705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b67b4d9-2422-41ec-9d57-396c6800d396 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:49.033545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:49.034692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.035188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.035663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.036480 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:49.038129 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:49.048405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:49.049433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 14 01:18:49.050373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:49.053280 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 456/912] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Sep 14 01:18:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:49.059393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:49.079348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:49.163502 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d947eaa1-cb4f-44a6-a738-d6987144ab61 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:49.198584 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:49.199199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.199457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.199726 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.200345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:49.200639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:49.202023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:49.207460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 457/913] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:49 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:49.213101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:49.270118 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:49.323153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d132042e-46e1-4883-abe1-04f976226408 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:49.357920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:49.358553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.358825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.359085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.359706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:49.360000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:49.361374 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c37a84d-e971-49ca-b69c-2034e462a062 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:49.367184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 457/914] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:49 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:49.372853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:49.470298 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:49.493156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c02edaf-3421-47ee-b1f1-cacaef0b2856 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:49.528251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:49.528976 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.529268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.529529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.530216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:49.531594 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06852aaf-000f-4ba1-98eb-7ad469bf5ba0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:49.554756 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 458/915] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:18:49 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:49.757526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb177d4-db0c-46c0-bc4e-f0ad6eb8a0ac None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.758134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb177d4-db0c-46c0-bc4e-f0ad6eb8a0ac None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.758401 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb177d4-db0c-46c0-bc4e-f0ad6eb8a0ac None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.759900 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 458/916] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:49.765688 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd020e9-3905-4b11-af50-8eba9bb528a6 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:49.766065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd020e9-3905-4b11-af50-8eba9bb528a6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:49.766408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd020e9-3905-4b11-af50-8eba9bb528a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:49.786873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-dcd020e9-3905-4b11-af50-8eba9bb528a6 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:49.787877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-dcd020e9-3905-4b11-af50-8eba9bb528a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:49.864130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 459/917] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:49 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:50.131179 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb177d4-db0c-46c0-bc4e-f0ad6eb8a0ac None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:50.243495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb177d4-db0c-46c0-bc4e-f0ad6eb8a0ac None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:50.258387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:50.280248 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:50.280854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.281127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.281389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.282090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:50.283524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed9c9f-597a-4854-b02f-38b4ad86c904 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:50.301763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 459/918] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:50 2022] POST /identity/v3/services => generated 241 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:18:50.313782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee751-e6b4-4c07-9cbc-8542eb3b9289 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.314722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee751-e6b4-4c07-9cbc-8542eb3b9289 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.314722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfee751-e6b4-4c07-9cbc-8542eb3b9289 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.316346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 460/919] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:50.322417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f400a4-f413-4a7d-9cdb-fb27e784dfa5 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.322850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f400a4-f413-4a7d-9cdb-fb27e784dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.323066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f400a4-f413-4a7d-9cdb-fb27e784dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.344384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d9f400a4-f413-4a7d-9cdb-fb27e784dfa5 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:50.345603 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d9f400a4-f413-4a7d-9cdb-fb27e784dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:50.434054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 460/920] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:50.669650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759790da-b826-4e6b-9f45-64ceed27b7e4 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.670016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759790da-b826-4e6b-9f45-64ceed27b7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.670233 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759790da-b826-4e6b-9f45-64ceed27b7e4 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.671872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 461/921] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:50.677071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.677355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.677624 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.696408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:50.697590 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:50.704180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759790da-b826-4e6b-9f45-64ceed27b7e4 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:50.773077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:50.777106 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 461/922] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:50.818755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759790da-b826-4e6b-9f45-64ceed27b7e4 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:50.854525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:50.855450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:50.855883 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:50.856296 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:50.857038 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:50.858485 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:50.867807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:50.868794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 14 01:18:50.869636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:50.873888 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 462/923] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:50.875349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:50.909036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:50.980116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6526dc0b-1c3c-4a29-ab4b-be561ab90923 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.013371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.014006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.014282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.014608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.015223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:51.015520 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.016887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.026111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 462/924] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:51.027979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.044383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.141031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:51.149474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2260c6d-3784-4e42-9d64-c56782fe59c6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.149832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb86676-8f78-4d29-aa9f-5a83be316162 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.183630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.184284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:51.184425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.184728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.185349 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.196045 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.196894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.197569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.197952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:51.198230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.199014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:51.199468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.200987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:51.203188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:51.204332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:51.205396 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:51.214387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 463/926] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:51.214387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.214860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 463/926] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:51.217497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.322940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e1e160-35ac-4789-b92c-7156c8b5b9db None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.357179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2906515-db0d-48d8-b83a-5adcab1a0dbf None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.358452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.359133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.359508 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.359886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.360579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:51.361666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.362444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.375970 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 464/927] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:51.377025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.395968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:51.402628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.404315 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.404741 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.405566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.406588 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.407990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.433354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 464/928] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:18:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:18:51.493817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2512dbf-854f-4458-919d-4cb535b2dd6d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.537799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.538795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.539222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.539636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.552960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.554807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:51.570924 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:51.572140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:51.573304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:51.573304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:51.573304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:51.575218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 465/929] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:51.586030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.642901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:51.702013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b30d5-e085-443c-8c0b-40a711bac308 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.740806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.741424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.741700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.741962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.742672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:51.743030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.744549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeb316-84cd-41ca-bd08-8626c191d300 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.757030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 465/930] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:51.763270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.807312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ace04a-7bfd-41f7-a51b-92decbfbdb7f None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.807586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ace04a-7bfd-41f7-a51b-92decbfbdb7f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.807908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ace04a-7bfd-41f7-a51b-92decbfbdb7f None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.809614 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 466/931] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:51.815996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.816376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.816850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.836291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:51.836938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:51.838400 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:51.876801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ed83b-3e9e-4caf-a75b-31bef771ab9a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:51.908646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 467/932] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:51.913358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:51.914261 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:51.914760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:51.915223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:51.923185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:51.924883 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:51.936382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:51.937375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:51.938362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:51.941301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 466/933] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:51.946790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:51.988604 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.050530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191a08e5-61e1-42b7-a275-23cee70f7994 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.086630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.087545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.088034 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.088496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.089423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:52.089934 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.091595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.110069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 468/934] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:52.122050 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.188473 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.221835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d05f8a-b6e0-40b6-84e2-dacfa7593286 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.247529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.260698 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.261651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.262133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.262583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.274382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.276116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:52.291087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:52.292066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:52.292984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:52.292984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:52.292984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:52.299615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 467/935] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:52.300838 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.301843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.311101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafef63-c9fa-4a3d-bed6-906fccf0fe6d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.358069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.358712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.358942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.359153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.359761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.361659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:52.373979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:52.375019 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:52.375882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:52.378133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 469/936] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:52.381421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.415993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15091155-4923-43c7-ba5d-1ffa654e0f16 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.453381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.454462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.454938 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.455399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.456332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:52.456843 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.458549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.476937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 468/937] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:52.478515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.509102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108a526-bc24-433a-9736-d20f3e0cbde9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.518896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.550901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.551509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.551782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.552050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.552680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:52.552978 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.555040 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.566098 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 470/938] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:52.567684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.592782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688326c2-0b73-474d-9d61-ef88e28f47bf None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.598602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.633613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.634602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.635208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.635971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.644051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.645735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-550d8ca6-2072-4e61-b491-f265cabbbfe3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:52.660450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:52.661428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:52.662547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:52.670199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 469/939] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:52.671630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb461-2037-42a0-9efa-1095d23f6c65 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.671925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb461-2037-42a0-9efa-1095d23f6c65 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.672202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbb461-2037-42a0-9efa-1095d23f6c65 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.677815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 470/940] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:52.677815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbb461-2037-42a0-9efa-1095d23f6c65 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.678048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fb5847-be1f-402e-a1da-9bf9d29055b6 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.711358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.711966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.712239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.712507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.713135 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:52.713433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.714828 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe721fe-0be8-4cfa-9f35-d383505f1d89 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.719223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 471/941] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:52.720446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.720729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.720996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.746043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:52.746043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:52.813655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cbb461-2037-42a0-9efa-1095d23f6c65 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:52.818343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.823268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 472/942] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:52.857679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:52.858414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:52.858695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:52.858967 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:52.859737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:52.860093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:52.861885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:52.875975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 471/943] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:52.881586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:52.904621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:52.988576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fba435-259d-4fef-9780-99e9c3575150 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.027680 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.028661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.029260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.029710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cf3b078d5a134266a19c889b477fb60f/roles/16b8db405fe9477e8959d74aed6537e4` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.032166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=cf3b078d5a134266a19c889b477fb60f, role_id=16b8db405fe9477e8959d74aed6537e4)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.033793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.051532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 473/944] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:18:52 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/cf3b078d5a134266a19c889b477fb60f/roles/16b8db405fe9477e8959d74aed6537e4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:18:53.097265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:53.212785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03dcf8-da54-4539-b2f9-683319fd6bc6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.253117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.253753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.254058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.254327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.254908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.256271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:53.266137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:53.267148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:53.267975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 472/945] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:53.273751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:53.362547 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:53.387537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2bb3d-6f1b-4f08-9472-6f1f2b7524f0 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.421907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.422929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.423422 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.423896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.424757 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:53.425253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.426851 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.432185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 474/946] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:53.442045 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:53.465082 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:53.554291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab32a8-e0a7-432d-b06b-dc45c81c839a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.589791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.590463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.590740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.591010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.591636 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:53.591935 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.593310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2fb1c9-2727-4664-a7bd-67ed624b8f28 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.598685 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 473/947] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:53.610310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:53.723237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff061d8c-3370-4532-96a1-d19477228390 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.758107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.758769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.759041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.759305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.760009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] RBAC: Adding query filter params (interface=public, service_id=f64eecdd2add4a6a86c6260300e3b089, region_id=RegionOne) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:53.760310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.761676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.767179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 475/948] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:53 2022] GET /identity/v3/endpoints?service_id=f64eecdd2add4a6a86c6260300e3b089&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:53.769659 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14271-0d8f-424f-8777-c4833bd57312 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.774037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14271-0d8f-424f-8777-c4833bd57312 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.774037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c14271-0d8f-424f-8777-c4833bd57312 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.774037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 474/949] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:53.775274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:53.776806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.777146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.777486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.785147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:53.797901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:53.799214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:53.875109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 475/950] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:53.902280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4e291-a434-47af-b5ba-21b369a9fdbb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:53.937334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:53.938217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:53.938632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:53.939394 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:53.940210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:53.941870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:53.948546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 476/951] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:53 2022] GET /identity/v3/services => generated 1497 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:18:54.143490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:54.160668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:54.252787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f599b72c-7d72-43bd-af26-47fa1c6d8cfe None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:54.287894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:54.288854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:54.289338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:54.289811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:54.290670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:54.292296 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:54.295220 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: WARNING keystone.server.flask.common [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 14 01:18:54.315548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.sql.core [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:18:54.315548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 01:18:54.315548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [parameters: {'id': '1d0a93f408f8464aaf93d520708e92b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 01:18:54.315548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81414) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d0a93f408f8464aaf93d520708e92b8.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d0a93f408f8464aaf93d520708e92b8. Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:54.334328 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 14 01:18:54.335182 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application yield resource Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise self.value Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application session.commit() Sep 14 01:18:54.336577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:18:54.337457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:18:54.338790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:54.339555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [parameters: {'id': '1d0a93f408f8464aaf93d520708e92b8', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:54.341470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:54.343416 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d0a93f408f8464aaf93d520708e92b8. Sep 14 01:18:54.345438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:54.348443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 476/952] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Sep 14 01:18:54 2022] POST /identity/v3/projects => generated 262 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:54.355053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:54.427682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:54.464681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fe59d-f6c3-4c9d-80bc-d6baa1e10eae None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:54.503471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:54.504115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:54.504387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:54.504653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:54.511711 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:54.513165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-c7622c7f-099a-4fd6-9eb9-d767e911febf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:54.524581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:54.525556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:54.526538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 477/953] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:54.535531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:54.609213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:54.642358 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5703b48-f878-47b5-bcae-455c225273f7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:54.676952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:54.678023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:54.679452 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:54.679982 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:54.680879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:54.681426 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:54.683299 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13631c0-eb7b-429c-bc79-cfa38ddf4354 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:54.704620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 477/954] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:55.018535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410a59-bc5a-4bf4-807f-a81ccc2d2c95 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.018935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410a59-bc5a-4bf4-807f-a81ccc2d2c95 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.019111 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52410a59-bc5a-4bf4-807f-a81ccc2d2c95 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.020703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 478/955] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:55.026196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287ae0-2961-44b5-a4a4-2497f20d6824 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.026718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287ae0-2961-44b5-a4a4-2497f20d6824 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.027189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48287ae0-2961-44b5-a4a4-2497f20d6824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.046119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-48287ae0-2961-44b5-a4a4-2497f20d6824 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:55.047925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-48287ae0-2961-44b5-a4a4-2497f20d6824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:55.118132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 478/956] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:55.289333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a01f4-9bdc-45e4-bdb0-b7bcc90eea2c None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.289947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a01f4-9bdc-45e4-bdb0-b7bcc90eea2c None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.290462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9a01f4-9bdc-45e4-bdb0-b7bcc90eea2c None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.292266 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 479/957] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:55.297940 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.298256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.298530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.317491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:55.318728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:55.374210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:55.382190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a01f4-9bdc-45e4-bdb0-b7bcc90eea2c None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:55.385578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 479/958] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:55.483926 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9a01f4-9bdc-45e4-bdb0-b7bcc90eea2c None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:55.512765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:55.518866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:55.519640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.520105 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.520371 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.521029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:55.522590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:55.546269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 480/959] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:18:55 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:18:55.649184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:55.756262 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016557d5-9860-429b-be0e-47f99c9dae44 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:55.796786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:55.797457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.797779 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.798070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.808625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:18:55.813431 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:55.814775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:55.815778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 480/960] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:18:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:55.821396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:55.920955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14857389-5918-4cde-afcd-d98ce33b34dd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:55.954590 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:55.955251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:55.955524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:55.955789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:55.956483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:55.956782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:55.958244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:55.966770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 481/961] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:18:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:55.973124 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.019592 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.073443 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ad32-9595-48e1-b6cc-66f5dadf5849 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.111643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.112442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.112759 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.113026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.124745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.126266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-92bc75df-1ebe-42cc-996d-adc30dc68f47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:56.143558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:56.144589 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:56.145486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:56.145486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:56.145486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:56.145486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 481/962] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:56.150830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.204219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.252183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ce2-033d-4e4e-8895-6ddca8e6a766 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.252544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ce2-033d-4e4e-8895-6ddca8e6a766 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.252781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d5ce2-033d-4e4e-8895-6ddca8e6a766 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.254430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 482/963] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:18:56.259169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc784d4-f6e7-4a7a-8c6f-6377b698b8a2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.260861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.261119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.261395 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.279436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:56.280921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:56.301748 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.302530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.302806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.303074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.304006 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:56.304304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.305670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.321185 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 482/964] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:56.326844 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.362332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 483/965] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:56 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:18:56.428184 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e61cf1-397e-4987-988e-c06552d93f1a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.441866 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.463324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.464003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.464283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.464596 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.471844 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.473347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:56.485048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:56.486497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:56.487479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 483/966] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:56.492486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.522284 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.598546 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63c8637-347b-4383-9bc7-7d9f77409d82 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.632593 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.633207 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.633480 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.633788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.635292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.639230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:56.639541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.643467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.662874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 484/967] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:56.668668 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.760922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.770613 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b0cc1-bd2a-4cac-9f78-dadb1bad14bb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.777757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7941588-5b2e-4327-b7af-74ec4e23a6fc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:56.807810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.808917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.809470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.809933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.810872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.812682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.813833 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:56.814823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:56.815093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:56.815398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:56.824507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:56.825591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:18:56.826481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:56.829327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 484/968] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:56.836253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:56.890610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:56.902858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:56.904944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:56.980732 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd166fbf-e675-44ab-9a63-31f853365683 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.010601 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.027927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 485/969] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:18:57.028687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.029445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.029713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.030000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.030638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:57.030936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.032305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.033646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.038810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 485/970] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:57.043968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.144122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c87dfd-dd24-4e15-98a9-d2a7656882c1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.151175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.155148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5012df2b-1749-4156-8e44-de074bf2fcd8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.180801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.181588 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.181859 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.182153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.188661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.189664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.189937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.190291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.190937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:57.191224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.192601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbb321-bd48-4ad5-aaab-206314049987 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.193176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.195012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.197341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 486/971] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:57.198511 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c729f0-6dac-4014-8b8e-d6c0e30d42c5 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.198790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c729f0-6dac-4014-8b8e-d6c0e30d42c5 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.199059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c729f0-6dac-4014-8b8e-d6c0e30d42c5 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.200249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 487/972] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:57.205062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c729f0-6dac-4014-8b8e-d6c0e30d42c5 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-42ac1349-dee9-4897-8b02-01ff66236815 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:18:57.211962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:57.212912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:18:57.213792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:18:57.213792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:18:57.213792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.219506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 486/973] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:57.219506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.219506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.219506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.235289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:57.236551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:57.266081 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.307283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 487/974] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:57 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:57.308008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.323356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c729f0-6dac-4014-8b8e-d6c0e30d42c5 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.373036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.374225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.375048 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.375746 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.376636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.378673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.404477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 488/975] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:18:57 2022] POST /identity/v3/endpoints => generated 349 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:18:57.424908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd1a3a-8cb3-4567-9bed-2d5165c44462 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.473855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.474558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.474831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.475097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.475739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:57.476035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.477542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.491743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 488/976] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:57.497996 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.524237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.577957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.609568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.641096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238dccc-710d-405f-b187-daa05401cbd7 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.682352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.682916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.683192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.683462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.690776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.692247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:18:57.703797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:57.704913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:18:57.706018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 489/977] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:57.707667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3c3d6-69c9-4298-a243-ebba1f8d87a8 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.711251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.750483 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.751465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.751946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.752576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.754399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.755893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.771868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.sql.core [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:18:57.771868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 01:18:57.771868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [parameters: {'user_id': '16eb708899124423bec07d690d8948ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 01:18:57.771868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81414) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:57.781453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:57.782578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application yield resource Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application next(self.gen) Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.session.rollback() Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.force_reraise() Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise self.value Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application session.commit() Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 14 01:18:57.783528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._prepare_impl() Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.session.flush() Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._flush(objects) Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application compat.raise_( Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application flush_context.execute() Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rec.execute(self) Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 14 01:18:57.784559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 14 01:18:57.785467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result = self._query(query) Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application conn.query(q) Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application result.read() Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 14 01:18:57.786428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [parameters: {'user_id': '16eb708899124423bec07d690d8948ea', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:57.788054 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 14 01:18:57.789655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 489/978] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Sep 14 01:18:57 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:18:57.791649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.833211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.861161 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f8b12-4f82-4162-84e9-2aa83032af76 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.896355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.897052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.897332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.897599 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.898366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:57.898666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.899554 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:57.900096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d1a19a-beb6-4d0d-860c-a4d35a29d2de None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.913193 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 490/979] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:57.914938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7fb4-58e2-47f2-9549-ba7e8f515638 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.914938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7fb4-58e2-47f2-9549-ba7e8f515638 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.916709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0a7fb4-58e2-47f2-9549-ba7e8f515638 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.922174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 491/980] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:57.922174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a7fb4-58e2-47f2-9549-ba7e8f515638 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:57.928861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b329e87-f3bb-4452-85c5-79aa9b1f6e43 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:57.966642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:57.968686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:57.968961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:57.969226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:57.978550 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:57.980189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-08875599-c7b3-40ef-954d-815b82bb8ee4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:18:57.994439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:57.995600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 14 01:18:57.996766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:58.000457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 490/981] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:58.001812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.002112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.002376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.021775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:58.023316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:58.029931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0a7fb4-58e2-47f2-9549-ba7e8f515638 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.066910 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:58.067544 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.067819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.068081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.070084 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] RBAC: Adding query filter params (role.id=16b8db405fe9477e8959d74aed6537e4, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=cf3b078d5a134266a19c889b477fb60f) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:58.070084 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:58.070561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:58.081801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 492/982] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:18:57 2022] GET /identity/v3/role_assignments?user.id=cf3b078d5a134266a19c889b477fb60f&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=16b8db405fe9477e8959d74aed6537e4 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:58.082709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:58.094861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:58.108378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 491/983] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:58 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:18:58.203957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5230cccd-f144-4ec8-bca1-812877e08610 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.240130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:58.241060 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.241507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.241944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.242893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:58.243368 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:58.245009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:58.284663 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 493/984] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Sep 14 01:18:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:58.380981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:58.467153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:58.491629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06f735b-f314-49ec-9a50-8405668aa55e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.526759 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:58.527492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.527830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.528177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.528810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:58.530322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:18:58.539420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:18:58.540658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:18:58.541683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:18:58.543787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 492/985] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:18:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:18:58.548890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:58.616390 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:58.654700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4148aca4-e289-4a5c-8aca-b60b5c5afe89 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.689739 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:58.690489 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.690891 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.691229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.691925 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:58.692286 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:58.693729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:58.699031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 494/986] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:18:58.704627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:58.804532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fecc9a-ccdd-496b-8697-7a1782d5bcd9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.837205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:58.837805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:58.838099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:58.838364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:58.838983 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:58.839275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:58.840634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:58.848354 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 493/987] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:18:58 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:58.854164 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:58.963734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafcffcc-11bd-4aa1-b46b-a464adb23922 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:58.990090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:59.000639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:59.001320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:59.001652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:59.002001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:59.002832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] RBAC: Adding query filter params (interface=public, service_id=3aaa63929bfa421e9731af8d3c175112, region_id=RegionOne) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:18:59.003196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:59.004627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a539d12-351a-44fa-87d1-25ecf4b58a8b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:59.009871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 495/988] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:18:58 2022] GET /identity/v3/endpoints?service_id=3aaa63929bfa421e9731af8d3c175112&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:18:59.015496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:18:59.019499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:59.124879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b6b2f-e356-4068-895e-b97d1d63b017 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:18:59.159107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:18:59.159709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:59.159975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:59.160252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:59.160830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:18:59.162230 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933157d-5453-41a7-a0c2-01aee9ad7814 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:18:59.168652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 494/989] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:18:59 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:18:59.665320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead220dd-813c-4162-b1be-d0c5bba9182c None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:59.665902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead220dd-813c-4162-b1be-d0c5bba9182c None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:59.665902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead220dd-813c-4162-b1be-d0c5bba9182c None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:59.667222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 496/990] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:18:59.672904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:18:59.673152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:18:59.673446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:18:59.692449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:18:59.693620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:18:59.757402 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:18:59.766154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 495/991] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:18:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:00.038134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead220dd-813c-4162-b1be-d0c5bba9182c None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.150937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead220dd-813c-4162-b1be-d0c5bba9182c None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.184562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.185188 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.185456 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.185722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.186332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.187711 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-44abdafe-61cc-4a81-b438-5afd8bc74190 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:00.197067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:00.198310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:00.199209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.203464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 497/992] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:00.204492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.307887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f96518-47ec-45da-9e32-3b7cd90a47da None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.342618 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.343229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.343497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.343763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.344381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:00.344674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.346043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.354056 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 496/993] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:00.354222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a0cf-2ade-4a4c-9f88-c0c3b6f2aa47 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.354222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a0cf-2ade-4a4c-9f88-c0c3b6f2aa47 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.354222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c4a0cf-2ade-4a4c-9f88-c0c3b6f2aa47 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.355127 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 498/994] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:00.356860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.359789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.360066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.360315 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.378407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:00.379601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:00.420176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:00.429249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:00.446882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 499/995] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:00 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:00.463341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20acfd-17bd-401c-b2d1-f5c30c810a94 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.497595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.498468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.498918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.499386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.500178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:00.500666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.502217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.507224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 497/996] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:00.513279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.611530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08238549-355c-4c31-8ea6-2cbcc972bd0d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.644783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.645389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.645661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.645941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.646656 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] RBAC: Adding query filter params (interface=admin, service_id=f64eecdd2add4a6a86c6260300e3b089, region_id=RegionOne) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:00.646960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.648333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.659633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 500/997] 10.222.0.26 () {56 vars in 1249 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/endpoints?service_id=f64eecdd2add4a6a86c6260300e3b089&interface=admin®ion_id=RegionOne => generated 193 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:00.659794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.705685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:00.758517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:00.766410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe147d3-ed27-4491-8bf8-7d437a586579 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.792650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:00.802506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.803219 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.803500 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.803971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.804469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.805837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.812299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec997e-5509-43df-a91a-97bf8f552f21 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:00.813037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 498/998] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services => generated 1727 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:19:00.851778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:00.852426 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:00.852696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:00.852958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] PATH_INFO: `/v3/services/image` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:00.853549 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:00.854994 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:00.864867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:00.865913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:00.866787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:00.870435 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 501/999] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:00.872198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:01.011674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757244a7-bdf8-4839-9198-ab5e588921e7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:01.047835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:01.048450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.048725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.048992 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.049620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] RBAC: Adding query filter params (name=image) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:01.049919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:01.051318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c7b1f2-33f7-4811-958e-ca8fc87eee9a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:01.060840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 499/1000] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:01.062431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:01.171685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbdb865-2fce-426f-a601-2a160bbb024a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:01.185933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:01.211115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:01.211744 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.212017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.212284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.212928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:01.213232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:01.214668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf3df5a-f935-4a45-9c58-73a4c98478b3 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:01.226072 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 502/1001] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:01 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:01.314925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0075b06-678b-46a4-a4fe-f8f142e86605 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.315502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0075b06-678b-46a4-a4fe-f8f142e86605 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.315502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0075b06-678b-46a4-a4fe-f8f142e86605 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.317197 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 500/1002] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:01.323261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.323571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.323811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.343206 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:01.344439 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:01.419963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 503/1003] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:01 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:01.687663 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0075b06-678b-46a4-a4fe-f8f142e86605 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:01.749180 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:01.797237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0075b06-678b-46a4-a4fe-f8f142e86605 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:01.831590 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:01.832229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.832461 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.832758 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.833321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:01.834721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:01.843646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:01.844755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:01.845605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 501/1004] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:19:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:01.850872 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:01.953428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ea5f4-a3c7-49e4-9d00-35f6b98c86fd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:01.961455 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:01.989481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:01.990126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:01.990401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:01.990664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:01.991292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] RBAC: Adding query filter params (name=compute) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:01.991594 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:01.992959 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:02.002555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 504/1005] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:19:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:02.003638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:02.115278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e456a62a-af79-4e27-8613-de7592a0179d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:02.132822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:02.150987 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:02.151594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:02.151863 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:02.152126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:02.152730 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:02.153026 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:02.154465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf6b4c-728a-4622-9b57-2826702f0356 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:02.159405 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 502/1006] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Sep 14 01:19:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:02.165043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:02.264005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a393614-b1a5-4bea-9cf7-3e2d467c20ed None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:02.297314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:02.297918 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:02.298214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:02.298484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:02.299147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:02.300654 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f1309f-4580-4f2e-a2f0-39abec3d3a22 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:02.320761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 505/1007] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:19:02 2022] POST /identity/v3/endpoints => generated 334 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:02.957652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf6d19-5f40-490b-aafa-ca96c02e670a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:02.958266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf6d19-5f40-490b-aafa-ca96c02e670a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:02.958266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf6d19-5f40-490b-aafa-ca96c02e670a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:02.959738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 503/1008] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:02.964994 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2745ee-be4c-47b8-aee0-66d6ee1b39b2 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:02.965411 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2745ee-be4c-47b8-aee0-66d6ee1b39b2 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:02.965584 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2745ee-be4c-47b8-aee0-66d6ee1b39b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:02.984443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-4a2745ee-be4c-47b8-aee0-66d6ee1b39b2 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:02.985642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4a2745ee-be4c-47b8-aee0-66d6ee1b39b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:03.006972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:03.063826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 506/1009] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:02 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:03.327540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf6d19-5f40-490b-aafa-ca96c02e670a None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:03.357646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5798c80-9f27-4ed3-869f-4aa31ad0dcea None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.357887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5798c80-9f27-4ed3-869f-4aa31ad0dcea None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.358372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5798c80-9f27-4ed3-869f-4aa31ad0dcea None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.360029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 507/1010] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:19:03.364836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.365126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.365378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.383671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:03.384871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:03.440570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf6d19-5f40-490b-aafa-ca96c02e670a None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:03.453488 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 508/1011] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:03 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:03.475058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:03.475691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.475965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.476237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.476805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:03.478196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:03.487120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:03.488072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 14 01:19:03.488908 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:03.493311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 504/1012] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Sep 14 01:19:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:03.494165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:03.576964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:03.596716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6cf58-aeb4-4525-86a3-1cde398b52da None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:03.630962 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:03.631570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.631840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.632109 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.632735 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:03.633039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:03.634447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:03.639518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 509/1013] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:19:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:03.645355 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:03.703157 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:03.716712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:03.759684 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772e73ae-a775-40c6-9b37-04ae918d4939 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:03.792783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:03.793384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.793692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.793919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.794560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:03.794852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:03.796326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:03.806036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 505/1014] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Sep 14 01:19:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:03.807342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:03.818049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab5117-7a91-46af-93df-26db3a4632bf None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:03.851495 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:03.852036 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.852316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.852585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.853250 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:03.854665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2747eeb5-2957-4115-b1a6-90429069b54f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:03.871487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 510/1015] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:19:03 2022] POST /identity/v3/services => generated 242 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:19:03.915321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b5c9e4-5a2a-4b96-8462-63c0c7dbbd50 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:03.927451 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:03.961148 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:03.961811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:03.962129 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:03.962292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:03.963000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:03.964361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ad993-4358-4682-81fb-1426fe91922b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:03.984014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 506/1016] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:19:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:04.550554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ba1c-8c1e-4b6b-94e2-6f2c0487115d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:04.551090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ba1c-8c1e-4b6b-94e2-6f2c0487115d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:04.551090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6ba1c-8c1e-4b6b-94e2-6f2c0487115d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:04.552625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 511/1017] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:04.557810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:04.558133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:04.558406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:04.577542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:04.578796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:04.657471 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 507/1018] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:04 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:04.924627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6ba1c-8c1e-4b6b-94e2-6f2c0487115d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:04.934032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:05.032602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6ba1c-8c1e-4b6b-94e2-6f2c0487115d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.066773 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.070585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.070585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.070585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.082346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:05.087177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:05.088114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:05.088973 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:05.090234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 512/1019] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:19:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:05.094348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:05.191089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:05.200821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204b00af-af8d-4c47-9756-ab4ae51b800e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.235532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.236145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.236418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.236764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.237544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:05.237845 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:05.239245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:05.252160 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 508/1020] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:05.253435 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:05.321674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:05.359630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f82822e-536b-4c48-9a42-7ff76db2ac6c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.396852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.397510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.397785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.398051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.409124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:05.410620 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:05.425035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:05.426000 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:05.431927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:05.431927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:05.431927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:05.431927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 513/1021] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:05.433540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:05.535720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aef8b01-cfa5-493a-9809-f63cf5d16e62 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.569393 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.570028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.570288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.570551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.571188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:05.571485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:05.572843 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:05.585723 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 509/1022] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:05.591259 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:05.693319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f145a3e-5e28-4b70-890c-4d39b554263a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.710821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:05.729927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.730578 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.730845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.731105 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.738664 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:05.740106 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:19:05.751155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:05.752143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:05.753208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:05.754329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 514/1023] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:05.758352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:05.775387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:05.864664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2114c47-d681-4c50-90be-01f5ee03a28d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:05.900413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:05.901029 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:05.901308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:05.901577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:05.902333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:05.902639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:05.904068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a933f5-9a6e-447a-bab6-e107f0818a83 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:05.928278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 510/1024] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:05.929693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.034876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc84d33b-04e1-4858-8d4b-884508e4aad2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.038904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe46b-81e2-4293-9aa2-d5ee01f9965b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.039270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe46b-81e2-4293-9aa2-d5ee01f9965b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.039439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6fe46b-81e2-4293-9aa2-d5ee01f9965b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.041060 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 511/1025] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:06.046854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca0c45-594c-411f-9416-e54be5048576 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.047190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca0c45-594c-411f-9416-e54be5048576 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.047683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca0c45-594c-411f-9416-e54be5048576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.067425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2cca0c45-594c-411f-9416-e54be5048576 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:06.078355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.078613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2cca0c45-594c-411f-9416-e54be5048576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:06.078966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.079240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.079506 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.126132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.150851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 512/1026] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:06.151368 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.152693 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.164375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431afc-5eeb-46ac-bfed-756d1390358a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.164925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431afc-5eeb-46ac-bfed-756d1390358a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.165195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af431afc-5eeb-46ac-bfed-756d1390358a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.166585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 513/1027] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:06.171631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65c7a4-4206-440c-b935-242640999379 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.171914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65c7a4-4206-440c-b935-242640999379 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.172224 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e65c7a4-4206-440c-b935-242640999379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.180670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.192202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-1e65c7a4-4206-440c-b935-242640999379 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:06.193390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-1e65c7a4-4206-440c-b935-242640999379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:06.253601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 515/1028] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:05 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:06.259226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 514/1029] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:06.259734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.337268 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.359322 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef55cd-dccd-4209-a6ad-e0688f8fb8ab None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.392611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.393186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.393468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.393763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.404067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.405520 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:06.420451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:06.421388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:06.422342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:06.422342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:06.422342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:06.422342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 516/1030] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:06.427259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65c7a4-4206-440c-b935-242640999379 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.430352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.527827 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e65c7a4-4206-440c-b935-242640999379 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.538437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49631d5b-5210-45ce-b2b5-19fbdf681731 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.557321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.562386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.563003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.563294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.563559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.564560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:06.564860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.566292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.567142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.573525 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.574184 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.574461 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.574735 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] PATH_INFO: `/v3/services/image` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.575319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.576678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.578183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 515/1031] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:06.579406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:06.585510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:06.590155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:06.591047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 517/1032] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:06.592854 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.680133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf00bc0-2e13-4331-a2ba-193ca38065d4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.687257 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95915cd8-7f0a-4180-9803-a1610363c995 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.713820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.714505 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.714795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.715058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.716273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.717708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.719334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 516/1033] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:06.720090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.722419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.722983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.723381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.723673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.731053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.732523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:19:06.744003 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:06.745049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 518/1034] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:06.745970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.787776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.805426 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:06.829298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78153c21-3740-46ae-bc44-cc848f89afa5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.849780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15539af9-17ce-4bb4-9bfc-0a4b0a61b333 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:06.862427 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.863032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.863302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.863561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.864176 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] RBAC: Adding query filter params (name=image) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:06.864464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.865821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.870572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 517/1035] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:06.871331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.886943 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:06.887553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:06.887824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:06.888092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:06.888725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:06.890207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:06.918120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 519/1036] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:19:06 2022] POST /identity/v3/projects => generated 311 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:06.918931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:06.976978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedf597a-4d93-42ef-a92c-c227e9803278 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:07.004975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:07.019541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:07.020495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:07.020925 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:07.021208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:07.021906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:07.022322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:07.023901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:07.026320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ed93-8ba6-406d-9ff9-a8225d298a0b None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:07.034470 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:07.044944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 518/1037] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:07.047199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:07.080491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:07.081163 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:07.081451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:07.081701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:07.082380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] RBAC: Adding query filter params (type=image) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:07.082957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:07.084130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:07.089686 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 520/1038] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:07.096143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:07.171010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b396b9a-c64c-461c-846b-ac8a98ca568f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:07.204326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fccc88-8432-426b-8cba-66b3cf8bc5df None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:07.208798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:07.209507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:07.209879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:07.210271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:07.211089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] RBAC: Adding query filter params (role.id=16b8db405fe9477e8959d74aed6537e4, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:07.211489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:07.213058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:07.218888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 519/1039] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:07 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=16b8db405fe9477e8959d74aed6537e4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:07.246714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:07.247245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:07.247521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:07.247792 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:07.248508 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] RBAC: Adding query filter params (interface=public, service_id=19b82df240ef4f4c83e7e9aebf5cfb34, region_id=RegionOne) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:07.248818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:07.250283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffa8082-7ce1-4f6c-87ab-7e23fbaa0643 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:07.256929 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 521/1040] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:19:07 2022] GET /identity/v3/endpoints?service_id=19b82df240ef4f4c83e7e9aebf5cfb34&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:07.262962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:07.320401 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:07.378543 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf309d66-464d-4cf9-bc03-d5438d520e0e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:07.416456 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:07.417072 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:07.417343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:07.417607 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:07.418237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:07.419608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf009eb5-38b3-4695-a7ee-75636d74bfed None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:07.426539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 520/1041] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:19:07 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 14 01:19:09.102201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4556-1420-4995-b2e5-fbad8d717396 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.102721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4556-1420-4995-b2e5-fbad8d717396 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.102721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c4556-1420-4995-b2e5-fbad8d717396 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.104124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 522/1042] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:09.109563 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084458-e576-423b-9636-30b6a9149b6c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.110038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084458-e576-423b-9636-30b6a9149b6c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.110121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b084458-e576-423b-9636-30b6a9149b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.128552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-4b084458-e576-423b-9636-30b6a9149b6c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:09.129740 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4b084458-e576-423b-9636-30b6a9149b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:09.202172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 521/1043] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:09.342876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decede4c-2396-4be9-8f7b-7cac3444965c None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.343064 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decede4c-2396-4be9-8f7b-7cac3444965c None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.343283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decede4c-2396-4be9-8f7b-7cac3444965c None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.344829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 523/1044] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:09.349901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4d79-a43c-4326-a809-2c30960f6622 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.350275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4d79-a43c-4326-a809-2c30960f6622 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.350552 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf4d79-a43c-4326-a809-2c30960f6622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.369263 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-edaf4d79-a43c-4326-a809-2c30960f6622 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:09.370517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-edaf4d79-a43c-4326-a809-2c30960f6622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:09.407956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:09.450041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 522/1045] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:09.467411 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decede4c-2396-4be9-8f7b-7cac3444965c None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:09.502933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:09.556919 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d447d-1a8c-4381-abb8-66e08937737d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.557264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d447d-1a8c-4381-abb8-66e08937737d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.557421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d447d-1a8c-4381-abb8-66e08937737d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.559004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 523/1046] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:19:09.564200 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.564481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.564752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.572305 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decede4c-2396-4be9-8f7b-7cac3444965c None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:09.583083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:09.584288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:09.607556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:09.608130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.608397 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.608671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.609779 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:09.611252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:09.613041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 524/1047] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:09.618433 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:09.652061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 524/1048] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:09.710965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:09.722002 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a515c9-8c64-412e-b23b-c90fd0353cfb None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:09.725573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:09.769869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:09.770516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.770788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.771055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.771707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:09.773177 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:09.794104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 525/1049] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:19:09 2022] POST /identity/v3/users => generated 285 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:19:09.827345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba1540-8d69-426a-820d-0bd3380cff1f None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:09.829564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:09.867117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:09.867765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:09.868040 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:09.868306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:09.880581 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:09.885540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:09.890273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:09.891188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 525/1050] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:19:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:09.893396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:09.929278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.003950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c822809c-6d23-4e58-a979-0a6319140a7e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.045360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed337f7-c584-4a47-98d7-1540b262e2da None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.046434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.047282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.047746 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.048025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.048697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:10.048997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.050507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.059581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 526/1051] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:19:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:10.065733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.087531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.088171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.088457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.088718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] PATH_INFO: `/v3/services/image` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.089153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.089336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.090704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:10.100027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:10.101035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:10.101906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 526/1052] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Sep 14 01:19:09 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:10.106942 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.135110 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.178288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b23481-993d-414b-84eb-146eb0d6e6ae None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.209850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20275ea5-3a52-4995-90c3-521126c897b4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.212212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.212843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.213119 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.213387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.224479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.225941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:10.240236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:10.241207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:10.242721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:10.242721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:10.242721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:10.242721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 527/1053] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:10.247004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.248460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.249093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.249412 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.249635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.250840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] RBAC: Adding query filter params (name=image) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:10.250840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.252079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.257347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 527/1054] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:10.262735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.332764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.357120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.364342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f66ac-da71-40f2-ad39-8442b3026984 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.368847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b768145-6819-45ff-8ec0-7783a0a5d5bc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.398291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.398813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.399079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.399354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.399995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:10.400295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.401662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.404948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.405591 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.405867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.406169 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.406798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] RBAC: Adding query filter params (type=image) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:10.407098 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.408428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.413957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 528/1055] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:10.414225 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 528/1056] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/services?type=image => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:10.419383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.423614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.519524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b881f-957c-4bd9-8218-0d16ffb9a6a2 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.531031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4d818-c567-4d8c-9deb-6775cd1a7313 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.548586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.554459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.555095 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.555367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.555640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.562876 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.564722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.566972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.567593 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.567867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.568140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.568811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.570200 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:19:10.578214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:10.579166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:10.580115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:10.584247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 529/1057] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:10.585254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.596419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 529/1058] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:19:10 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:19:10.696974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf766748-0b62-4a4e-8317-7c4e0ba3f542 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.742725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.743365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.743640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.743901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.744659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:10.744961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.746462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.766432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 530/1059] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:10.772156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.785898 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:10.874584 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dc394b-7116-4ec0-b066-7f2493e3afba None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:10.907006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:10.907634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:10.907915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:10.908193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:10.919058 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:10.920504 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:10.934680 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:10.935652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:10.936553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:10.936553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:10.936553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:10.936553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 530/1060] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:10.943151 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:10.989326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:11.043958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ee3c57-c322-4f1b-a65c-f1bb53452753 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:11.082186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:11.082832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:11.083112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:11.083369 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:11.084005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:11.084300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:11.085674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:11.097600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 531/1061] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:11.103188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:11.166024 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:11.206687 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa6942-c5fc-4086-9e2c-e68c87ff603e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:11.239513 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:11.240071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:11.240338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:11.240597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:11.247191 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:11.248585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:19:11.259814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:11.260711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:11.266790 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 531/1062] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:19:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:11.267283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:11.370238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e3ec91-f49f-4efd-b73e-da06564f1d22 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:11.377333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:11.406691 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:11.407299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:11.407566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:11.407827 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:11.408513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:11.408810 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:11.410517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dd874a-a941-4636-a5cf-79dd489ab9da None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:11.427556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 532/1063] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:19:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:11.428765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:11.531130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f1928e-c544-4763-b24f-a75d0b3785ea None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:11.553511 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:11.566571 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:11.567264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:11.567548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:11.567815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/16b8db405fe9477e8959d74aed6537e4` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:11.570014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=6f6158bf6948476a9244d0c28e1c8404, role_id=16b8db405fe9477e8959d74aed6537e4)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:11.571445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26df7c8-ac02-4222-91bc-60d9240164bb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:11.594013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 532/1064] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:19:11 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/6f6158bf6948476a9244d0c28e1c8404/roles/16b8db405fe9477e8959d74aed6537e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:19:12.007334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f572c1-ed47-4e83-a4ce-aa62bb78eb9d None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.007739 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f572c1-ed47-4e83-a4ce-aa62bb78eb9d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.007916 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f572c1-ed47-4e83-a4ce-aa62bb78eb9d None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.009318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 533/1065] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:12.014726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.015011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.015257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.034178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:12.035388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:12.117352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 533/1066] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:12.377308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f572c1-ed47-4e83-a4ce-aa62bb78eb9d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:12.487172 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f572c1-ed47-4e83-a4ce-aa62bb78eb9d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:12.521160 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:12.521780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.522073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.522334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.532527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:12.537450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:12.538385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:12.539269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 534/1067] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:12.544544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:12.645132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f107516-0561-4259-8b38-076f368f20d9 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:12.659130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:12.684766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:12.685395 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.685637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.685926 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.686630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:12.686897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:12.688326 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2c30be-d694-4f75-896a-27c921242814 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:12.698302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 534/1068] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:12.704187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:12.733315 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:12.784450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f428-a2ff-44f7-9c29-a49fa73bf977 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.784693 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f428-a2ff-44f7-9c29-a49fa73bf977 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.785055 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85f428-a2ff-44f7-9c29-a49fa73bf977 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.786726 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 535/1069] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:19:12.791513 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.791865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.792213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.810131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:12.810314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f8d13-3d0b-465b-be59-b302554b394c None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:12.811400 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:12.852168 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:12.852805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:12.853081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:12.853347 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] PATH_INFO: `/v3/users/ca86fa294ce04074a461eb16730eb45a` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:12.868234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] RBAC: Authorizing `identity:get_user(user_id=ca86fa294ce04074a461eb16730eb45a)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:12.869712 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:12.871582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 535/1070] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:12 2022] GET /identity/v3/users/ca86fa294ce04074a461eb16730eb45a => generated 285 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:12.877101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:12.894209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 536/1071] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:12.965997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:12.982703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde8055c-3ddd-4c04-9772-8ffb29a8fb54 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.016223 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.016824 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.017089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.017356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.084869 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.086358 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.159294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:13.168092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.190982 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 536/1072] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:19:13.197804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:13.263461 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea4c4d-8c7e-483e-a288-6f75164a42b9 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.266767 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.298350 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e23ff8d-191c-4b8e-abbb-d200d8fae903 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.300786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.301458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.301735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.302018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.334625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.335262 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.335531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.335796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.336965 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.338440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.340087 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 537/1073] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:13 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:13.346092 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:13.375279 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.376741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.399783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.448242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f922d603-7b3e-45e4-a195-9e522ef82abc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.478645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 537/1074] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:19:13.484085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:13.487356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.489535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.490178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.490452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.490716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.491471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user.id=ca86fa294ce04074a461eb16730eb45a) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:13.491775 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.493602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.500282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 538/1075] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:13 2022] GET /identity/v3/role_assignments?user.id=ca86fa294ce04074a461eb16730eb45a&scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:13.602209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21d87-d0ac-404b-bace-302fc16388a6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.643657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.644306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.644577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.644839 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.656496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.657964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-5d848480-ef97-4a2f-8b56-97eef8e4f0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:13.673914 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:13.675064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:13.675962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:13.675962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:13.675962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:13.678353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 538/1076] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:19:13.681108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:13.737130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a24a78-775b-485f-8a67-bc7685ddbcea None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.737428 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a24a78-775b-485f-8a67-bc7685ddbcea None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.737696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a24a78-775b-485f-8a67-bc7685ddbcea None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.739093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 539/1077] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:13.744063 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.744341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.744606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.763749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:13.764922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:13.797642 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c22dd47-6629-422e-8f5f-e4aec582304a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:13.815238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.819569 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:13.835245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:13.835856 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:13.836120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:13.836385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:13.837026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:13.837320 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:13.838682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 540/1078] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:19:13.844558 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:13.862126 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 539/1079] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:14.101382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:14.202324 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b942fb89-cb28-43ec-b213-d04192ac39c7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:14.234717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:14.235341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:14.235608 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:14.235878 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:14.246018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:14.250766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:14.251899 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 14 01:19:14.253178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:14.254313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 541/1080] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:14.257852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:14.358055 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230709b-c58f-47cb-9dbb-a881d34d7213 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:14.395087 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:14.395684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:14.395954 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:14.396229 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:14.396879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:14.397171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:14.398582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:14.406381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 540/1081] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:14.412481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:14.459384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:14.516322 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72934444-3421-4b4b-ae7f-07c5c200b092 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:14.550635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:14.551257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:14.551527 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:14.551792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:14.562027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:14.563486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:14.578557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:14.579606 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:14.580875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:14.580875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:14.580875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:14.580875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 542/1082] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:14.585660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:14.624985 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:14.684042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeaa76c-5bb7-4543-9627-73d597173fa9 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:14.717764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:14.718323 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:14.718598 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:14.718865 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:14.719502 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:14.719800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:14.721186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:14.733791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 541/1083] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:14.739697 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:14.845292 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:14.849105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b13cce7-d92c-4617-8eac-6ee0abfb375d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:14.884114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:14.884739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:14.885009 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:14.885272 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:14.892776 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:14.894374 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:19:14.905453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:14.906442 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:14.907780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 543/1084] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:14.912665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.010059 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:15.014189 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359956d4-2c6d-4a5b-8d47-6008ef3fd522 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:15.048344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:15.049030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.049303 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.049601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.050353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:15.050651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:15.052080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:15.070251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 542/1085] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Sep 14 01:19:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:15.076220 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.183287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97a62e-7331-42f5-b86f-acd0661bf96e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:15.217819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:15.218448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.218720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.218979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.245966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:15.287432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:15.288791 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:15.387598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 544/1086] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:15.393845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.503560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af64b53-1f4a-4861-8251-e1c71f4a69b5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:15.543380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:15.543924 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.544201 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.544446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.555519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:15.556966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-5ce371bc-7a6a-4224-a60f-37ae376a2799 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:15.572258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:15.573220 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:19:15.574190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:15.574190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:19:15.574190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:15.574190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 543/1087] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:15.579267 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.630147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8577-ece7-4cd2-af0f-aeb04d4c5356 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.630430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8577-ece7-4cd2-af0f-aeb04d4c5356 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.630725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9e8577-ece7-4cd2-af0f-aeb04d4c5356 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.632096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 544/1088] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:15.636928 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.637206 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.637510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.637894 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:15.656890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:15.658156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:15.660120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:15.680365 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e043658d-5fc1-4245-95cf-1fa5b584d22c None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:15.712490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:15.713093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.713366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.713631 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.714298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:15.714593 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:15.715941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:15.724104 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 545/1089] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:15.730013 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 545/1090] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:15.733333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.829692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b1886-9310-426a-a138-b66a941e7d6e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:15.864475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:15.865085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:15.865352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:15.865611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:15.872526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:15.873968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:15.875767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:19:15.885763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:15.886914 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:19:15.888008 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:15.891862 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 546/1091] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:15.893034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.990244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:15.994596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5a8f4f-7e24-49bb-af29-1eead82d18b1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.011762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:16.028600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.029266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.029541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.029806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.030521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:16.030822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.032290 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.049506 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 546/1092] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:16.050597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:16.091936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d52c9-9701-4782-a0d7-f6da1f698c0e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.124443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.125066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.125333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.125598 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.135886 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:16.140632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:16.141713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:16.142790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 547/1093] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:16.147743 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:16.156239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d8e9d6-3831-4028-8701-d9046d586745 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.190987 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.191587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.191860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.192125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.192844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] RBAC: Adding query filter params (role.id=16b8db405fe9477e8959d74aed6537e4, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=6f6158bf6948476a9244d0c28e1c8404) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:16.193145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.194617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.200167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 547/1094] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:16 2022] GET /identity/v3/role_assignments?user.id=6f6158bf6948476a9244d0c28e1c8404&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=16b8db405fe9477e8959d74aed6537e4 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:16.212836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:16.250332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7444ff4-9cf9-40d8-bfbd-a4c5ef939b9d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.288230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.288846 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.289114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.289376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.290083 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:16.290380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.291768 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.302912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 548/1095] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:19:16.309073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:16.356387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:16.416897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc3918-5aab-4e0d-ade0-62f1a002b0bc None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.450765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.451641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.451916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.452183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] PATH_INFO: `/v3/users/ca86fa294ce04074a461eb16730eb45a` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.453301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] RBAC: Authorizing `identity:get_user(user_id=ca86fa294ce04074a461eb16730eb45a)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.454760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.456646 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 548/1096] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:16 2022] GET /identity/v3/users/ca86fa294ce04074a461eb16730eb45a => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:16.463909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:16.570155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3938e744-b67e-48d2-83d5-a63ba7c69282 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.590921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:16.606809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.607372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.607641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.607907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.609033 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.610503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ecd49-7e25-41c1-9367-042a2e1f7272 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.612404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 549/1097] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:16 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:16.618517 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:16.720259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec117862-ea8c-4f08-9fc9-ec3e0883b486 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:16.731212 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:16.754815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:16.755476 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:16.755747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:16.756015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b/users/ca86fa294ce04074a461eb16730eb45a/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:16.757885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user_id=ca86fa294ce04074a461eb16730eb45a, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:16.759384 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2b91a4-ccc2-4e0a-834b-49cb43e7ed24 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:16.772949 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 549/1098] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:19:16 2022] PUT /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b/users/ca86fa294ce04074a461eb16730eb45a/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:19:18.882211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46cbfd-3036-42a6-b097-59441aa5f029 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:18.884871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46cbfd-3036-42a6-b097-59441aa5f029 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:18.884871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e46cbfd-3036-42a6-b097-59441aa5f029 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:18.884871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 550/1099] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:18.890734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:18.891190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:18.891295 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:18.917373 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:18.918604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:18.988873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 550/1100] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:19.250640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46cbfd-3036-42a6-b097-59441aa5f029 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:19.360424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e46cbfd-3036-42a6-b097-59441aa5f029 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:19.396703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:19.397307 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:19.397579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:19.397848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:19.408567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:19.413387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:19.414528 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:19.415785 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 551/1101] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:19.420782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:19.520184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:19.523195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7683711c-2809-4fb7-a4b5-acdb77208d02 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:19.557970 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:19.558604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:19.558873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:19.559137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:19.559794 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:19.560086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:19.561458 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:19.569869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 551/1102] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:19.576389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:19.634440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:19.695081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ffdaa5-cc03-47ab-af49-dad0e97da639 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:19.732425 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:19.733415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:19.733845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:19.734334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] PATH_INFO: `/v3/users/ca86fa294ce04074a461eb16730eb45a` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:19.736303 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] RBAC: Authorizing `identity:get_user(user_id=ca86fa294ce04074a461eb16730eb45a)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:19.738685 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:19.741175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 552/1103] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:19 2022] GET /identity/v3/users/ca86fa294ce04074a461eb16730eb45a => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:19.747052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:19.848682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0e99b1-5965-4b87-929c-fead407df8d5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:19.883037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:19.883630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:19.883904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:19.884168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:19.914536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:19.955635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:19.957096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:20.053911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 552/1104] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:20.060175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:20.172806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d50554-8b42-4f67-8f1e-97a158b443a8 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:20.191421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:20.211661 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:20.212281 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:20.212557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:20.212821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:20.214004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:20.215533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6d16f5-4cc1-47ab-97f0-1fbaf18bc89f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:20.217171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 553/1105] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:20 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:20.223536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:20.285560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:20.324750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d3c947-9f48-4620-8096-4e9f5d7a8dd0 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:20.359690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:20.360278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:20.360548 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:20.360810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:20.361525 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user.id=ca86fa294ce04074a461eb16730eb45a) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:20.361825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:20.363331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e5799-dd9e-42be-a4ca-ee7409491420 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:20.369188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 553/1106] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:20 2022] GET /identity/v3/role_assignments?user.id=ca86fa294ce04074a461eb16730eb45a&scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:22.516338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a7a3-6895-4cc4-b20e-781d0abbb549 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:22.516923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a7a3-6895-4cc4-b20e-781d0abbb549 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:22.516923 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64a7a3-6895-4cc4-b20e-781d0abbb549 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:22.518853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 554/1107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:22.525310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:22.525714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:22.525867 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:22.558423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:22.559644 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:22.631116 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 554/1108] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:22.897449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64a7a3-6895-4cc4-b20e-781d0abbb549 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:23.009406 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64a7a3-6895-4cc4-b20e-781d0abbb549 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:23.047154 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:23.047770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:23.048041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:23.048291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:23.049572 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:23.051061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782c775-88d6-4ea2-a246-6fa716e2d628 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:23.052609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 555/1109] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:23.059142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:23.120158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:23.168906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e4b964-4098-4446-b07f-9a26f147d835 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:23.204158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:23.204754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:23.205030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:23.205297 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:23.206027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:23.207478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec3f290-0182-470f-86f2-ddfd18739555 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:23.230749 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 555/1110] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:19:23 2022] POST /identity/v3/users => generated 286 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:25.379411 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ebe-5c91-45f1-adca-a695418e33d2 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:25.379979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ebe-5c91-45f1-adca-a695418e33d2 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:25.379979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fc9ebe-5c91-45f1-adca-a695418e33d2 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:25.381297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 556/1111] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:25.387748 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:25.388198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:25.388300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:25.409192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:25.410420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:25.487719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 556/1112] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:25.769245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc9ebe-5c91-45f1-adca-a695418e33d2 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:25.834950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:25.889465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fc9ebe-5c91-45f1-adca-a695418e33d2 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:25.929235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:25.930241 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:25.930606 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:25.930958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] PATH_INFO: `/v3/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:25.933214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] RBAC: Authorizing `identity:get_role(role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:25.935396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:25.937364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 557/1113] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:25 2022] GET /identity/v3/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 223 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:25.943958 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:25.988806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:26.048499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea2741d-2d27-42e3-b2e5-ea151165e758 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:26.085128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:26.085769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:26.086043 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:26.086339 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] PATH_INFO: `/v3/users/f83a7da075194277bcb5f9a873d2a546` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:26.099679 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] RBAC: Authorizing `identity:get_user(user_id=f83a7da075194277bcb5f9a873d2a546)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:26.101120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:26.102759 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 557/1114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:25 2022] GET /identity/v3/users/f83a7da075194277bcb5f9a873d2a546 => generated 286 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:26.109578 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:26.217012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021780cf-9190-4a73-beb7-65f8b8191035 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:26.236760 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:26.251763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:26.252381 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:26.252648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:26.252938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:26.320150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:26.321530 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:26.419784 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 558/1115] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:26.426474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:26.529266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:26.532674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fdf4e3-5fe0-4ae3-9fa5-9a2d03d1523f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:26.571638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:26.572201 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:26.572472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:26.572738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:26.573976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:26.575408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229ac68-73af-4be3-a494-c3b1462b87fc None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:26.577097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 558/1116] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:26 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:26.586458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:26.607855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:26.693749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47671-72a7-4ecb-be14-8657ade1699f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:26.729120 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:26.729716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:26.730004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:26.730278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:26.730992 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user.id=f83a7da075194277bcb5f9a873d2a546) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:26.731292 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:26.732717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9b4d63-50a1-452b-a9f5-9d3043630c0b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:26.738216 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 559/1117] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:26 2022] GET /identity/v3/role_assignments?user.id=f83a7da075194277bcb5f9a873d2a546&scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:28.837593 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc373-3419-4777-8716-fe1baf5e6082 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:28.838559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc373-3419-4777-8716-fe1baf5e6082 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:28.838559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5cc373-3419-4777-8716-fe1baf5e6082 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:28.840667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 559/1118] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:28.846950 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:28.847382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:28.847490 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:28.867591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:28.868798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:28.944131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 560/1119] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:29.219541 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc373-3419-4777-8716-fe1baf5e6082 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:29.334315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5cc373-3419-4777-8716-fe1baf5e6082 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:29.376062 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:29.376678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:29.376948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:29.377215 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] PATH_INFO: `/v3/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:29.378781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] RBAC: Authorizing `identity:get_role(role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:29.380222 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:29.381537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 560/1120] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:29 2022] GET /identity/v3/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 223 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:29.388014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:29.411232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:29.497829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037779fd-1fc4-4bea-9bc3-df76c9f77e41 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:29.534094 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:29.534707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:29.534974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:29.535239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] PATH_INFO: `/v3/users/f83a7da075194277bcb5f9a873d2a546` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:29.536475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] RBAC: Authorizing `identity:get_user(user_id=f83a7da075194277bcb5f9a873d2a546)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:29.537898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:29.539479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 561/1121] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:29 2022] GET /identity/v3/users/f83a7da075194277bcb5f9a873d2a546 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:29.545437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:29.580971 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:29.645430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6cb76c-08d9-47c7-be97-cb371819ab0b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:29.679932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:29.680540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:29.680808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:29.681073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:29.682313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:29.683728 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5ef4b2-2c04-430c-b4b9-98f8505a0f63 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:29.685348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 561/1122] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:29 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:29.691945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:29.780759 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:29.799627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58740110-d989-4bcb-ba8f-9bce9dc1a866 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:29.834335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:29.834964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:29.835233 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:29.835493 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b/users/f83a7da075194277bcb5f9a873d2a546/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:29.838031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] RBAC: Authorizing `identity:create_grant(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user_id=f83a7da075194277bcb5f9a873d2a546, role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:29.839466 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50997e-a066-4b5f-8b34-77fe07011560 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:29.854660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 562/1123] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:19:29 2022] PUT /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b/users/f83a7da075194277bcb5f9a873d2a546/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:19:31.956886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05578bc-196e-4c0c-8a36-224918afaa31 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:31.957657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05578bc-196e-4c0c-8a36-224918afaa31 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:31.957657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05578bc-196e-4c0c-8a36-224918afaa31 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:31.959121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 562/1124] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:31.964567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:31.965023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:31.965217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:31.984581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:31.985757 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:32.072131 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 563/1125] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:32.334404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05578bc-196e-4c0c-8a36-224918afaa31 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:32.458192 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05578bc-196e-4c0c-8a36-224918afaa31 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:32.460497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:32.492817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:32.493429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:32.493695 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:32.493957 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] PATH_INFO: `/v3/roles/a5d5ae5918264aa798b161b3f054ea13` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:32.495411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] RBAC: Authorizing `identity:get_role(role_id=a5d5ae5918264aa798b161b3f054ea13)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:32.496781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:32.498134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 563/1126] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:32 2022] GET /identity/v3/roles/a5d5ae5918264aa798b161b3f054ea13 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:32.504424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:32.592600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:32.624297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a386549f-2a00-402c-a4f7-6b909a0b50af None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:32.658814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:32.659428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:32.659697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:32.659970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] PATH_INFO: `/v3/users/f83a7da075194277bcb5f9a873d2a546` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:32.661132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] RBAC: Authorizing `identity:get_user(user_id=f83a7da075194277bcb5f9a873d2a546)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:32.662592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:32.664178 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 564/1127] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:32 2022] GET /identity/v3/users/f83a7da075194277bcb5f9a873d2a546 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:32.672671 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:32.783161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf4e93e-451a-4acd-bfdb-80499d3554b3 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:32.824046 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:32.824643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:32.824913 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:32.825177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:32.869325 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:32.898120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:32.899569 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:33.000637 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 564/1128] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 14 01:19:33.007610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:33.121052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd7ed5-60d8-4a0a-b561-afc63d82a20a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:33.155921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:33.156538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:33.156780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:33.157081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] PATH_INFO: `/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:33.158357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] RBAC: Authorizing `identity:get_project(project_id=09f6a8d8d76a4f8ca2e515c2c72b7f2b)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:33.159797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1beb400-9be5-4fb9-854a-a393d1097ebc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:33.161489 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 565/1129] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:33 2022] GET /identity/v3/projects/09f6a8d8d76a4f8ca2e515c2c72b7f2b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:33.167856 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:33.247409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:33.279012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c76a0c-b1da-48b7-ab0e-295784f2d196 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:33.314240 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:33.314834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:33.315105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:33.315366 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:33.316070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] RBAC: Adding query filter params (role.id=a5d5ae5918264aa798b161b3f054ea13, scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b, user.id=f83a7da075194277bcb5f9a873d2a546) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:33.316369 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:33.317814 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa544c-caa3-4217-a5c6-23618b798859 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:33.323587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 565/1130] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:33 2022] GET /identity/v3/role_assignments?user.id=f83a7da075194277bcb5f9a873d2a546&scope.project.id=09f6a8d8d76a4f8ca2e515c2c72b7f2b&role.id=a5d5ae5918264aa798b161b3f054ea13 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:35.462763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b908-112f-46b1-9c8c-e1e51b5e7ef8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:35.463673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b908-112f-46b1-9c8c-e1e51b5e7ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:35.463673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b05b908-112f-46b1-9c8c-e1e51b5e7ef8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:35.465775 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 566/1131] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:35.471245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:35.471682 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:35.471780 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:35.490715 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:35.491902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:35.560786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 566/1132] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:35.824028 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05b908-112f-46b1-9c8c-e1e51b5e7ef8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:35.838579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:35.945279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b05b908-112f-46b1-9c8c-e1e51b5e7ef8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:35.981497 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:35.982144 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:35.982414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:35.982676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:35.983911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:35.985351 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deef452-a13c-4eb9-b197-fd533c4dbf49 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:35.986941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 567/1133] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:35.994016 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:36.063175 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:36.099115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a4fe74-40c7-43ee-ae79-0c7d1d6a7bad None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:36.133247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:36.133836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:36.134134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:36.134394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:36.135034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:36.136450 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be61aa8f-ca59-4134-b030-921df3e379ec None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:36.167417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 567/1134] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:19:35 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:38.336341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb76e5-63e7-4b03-bc1d-c46813dfc060 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:38.337166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb76e5-63e7-4b03-bc1d-c46813dfc060 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:38.337166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eb76e5-63e7-4b03-bc1d-c46813dfc060 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:38.339449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 568/1135] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:38.344909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:38.345342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:38.345433 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:38.365343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:38.366563 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:38.434847 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 568/1136] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:38.701714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb76e5-63e7-4b03-bc1d-c46813dfc060 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:38.792985 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:38.819334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eb76e5-63e7-4b03-bc1d-c46813dfc060 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:38.855455 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:38.856069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:38.856339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:38.856607 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:38.857855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:38.859326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229e6639-c0b6-48df-a369-1fc1ae8239fc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:38.860884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 569/1137] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:19:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:38.867107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:38.920760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:38.971884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b989841d-3d57-417f-af31-c0f991f8715c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:39.008061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:39.008670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:39.008943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:39.009213 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:39.009859 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:39.011323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d986e-dd35-4428-bc94-526ae63a8c55 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:39.037625 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 569/1138] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:19:38 2022] POST /identity/v3/users => generated 286 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:41.194341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc36-dd3a-4b27-b2e2-c4354efde4ca None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:41.195243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc36-dd3a-4b27-b2e2-c4354efde4ca None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:41.195243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b2dc36-dd3a-4b27-b2e2-c4354efde4ca None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:41.197372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 570/1139] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:41.203912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:41.204594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:41.204769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:41.225783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:41.227020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:41.300209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 570/1140] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:41.575806 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2dc36-dd3a-4b27-b2e2-c4354efde4ca None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:41.698819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b2dc36-dd3a-4b27-b2e2-c4354efde4ca None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:41.737184 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:41.738150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:41.738556 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:41.738960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:41.749919 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:41.754752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:41.755850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:41.756980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 571/1141] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:41.762147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:41.797004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:41.864372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4b15d-1e9e-4bcb-be23-f35331332805 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:41.899516 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:41.900107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:41.900380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:41.900649 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:41.901307 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:41.901657 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:41.903104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:41.911153 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 571/1142] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:41.917766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:41.924980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:42.027569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f55cad-f72c-4809-a543-85beabf1ea65 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:42.065382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:42.066022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:42.066297 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:42.066564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] PATH_INFO: `/v3/users/1e245f7fc96942f08ab02f5714a134ab` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:42.080174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] RBAC: Authorizing `identity:get_user(user_id=1e245f7fc96942f08ab02f5714a134ab)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:42.081633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:42.083354 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 572/1143] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:41 2022] GET /identity/v3/users/1e245f7fc96942f08ab02f5714a134ab => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:42.089417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:42.193035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:42.199742 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9ef2b2-9fa5-4967-b985-5b27e45439c6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:42.237613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:42.238258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:42.238495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:42.238758 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:42.317204 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:42.318755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:42.410382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:42.420917 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 572/1144] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:42.429665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:42.493781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:42.537136 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be651bd-4479-41e0-8152-fb33e1845d38 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:42.573425 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:42.574039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:42.574346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:42.574610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] PATH_INFO: `/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:42.575764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] RBAC: Authorizing `identity:get_project(project_id=10ff00dd2cfb416f8a0ec01a679d0a6b)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:42.577199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94d2498-fba6-4ee5-a697-806845e8f67f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:42.578871 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 573/1145] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:42 2022] GET /identity/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:42.585710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:42.689398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b284fac4-5f1b-46ac-8991-8c2f838402ad None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:42.725294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:42.725878 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:42.726174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:42.726436 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:42.727144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=10ff00dd2cfb416f8a0ec01a679d0a6b, user.id=1e245f7fc96942f08ab02f5714a134ab) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:42.727439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:42.728852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971ffd2-782a-4151-810e-6225f258cb97 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:42.734228 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 573/1146] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:42 2022] GET /identity/v3/role_assignments?user.id=1e245f7fc96942f08ab02f5714a134ab&scope.project.id=10ff00dd2cfb416f8a0ec01a679d0a6b&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:44.827342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8c07-5fd6-4365-a907-b9557875c554 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:44.827909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8c07-5fd6-4365-a907-b9557875c554 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:44.827909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a8c07-5fd6-4365-a907-b9557875c554 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:44.829256 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 574/1147] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:44.835932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:44.836641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:44.836761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:44.867459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:44.869312 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:44.937604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:44.951167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 574/1148] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:45.221853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8c07-5fd6-4365-a907-b9557875c554 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:45.339257 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a8c07-5fd6-4365-a907-b9557875c554 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:45.368409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:45.375279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:45.375893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:45.376165 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:45.376426 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:45.386458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:45.391219 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:45.392238 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:45.393202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 575/1149] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:45.399288 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:45.501289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d2eac-b78d-479a-a51d-216dbf60fe78 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:45.537138 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:45.537830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:45.538239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:45.538597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:45.539386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:45.539784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:45.541274 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:45.549331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 575/1150] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:45.558625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:45.665663 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3ee471-68bb-40be-b6eb-80da78e388f3 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:45.701567 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:45.702205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:45.702474 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:45.702735 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] PATH_INFO: `/v3/users/1e245f7fc96942f08ab02f5714a134ab` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:45.703892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] RBAC: Authorizing `identity:get_user(user_id=1e245f7fc96942f08ab02f5714a134ab)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:45.705332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:45.706931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 576/1151] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:45 2022] GET /identity/v3/users/1e245f7fc96942f08ab02f5714a134ab => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:45.713526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:45.770093 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:45.822470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edd8cf0-db6c-4ff0-85dc-ad570b36da01 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:45.858549 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:45.859185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:45.859491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:45.859755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] PATH_INFO: `/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:45.860922 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] RBAC: Authorizing `identity:get_project(project_id=10ff00dd2cfb416f8a0ec01a679d0a6b)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:45.862380 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d67f80f-8718-4877-a57f-6695d0866332 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:45.863993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 576/1152] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:45 2022] GET /identity/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:45.870338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:45.932728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:45.977099 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc353e4-4b06-45fa-a299-a52a49ced82e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:46.014017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:46.014666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:46.014932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:46.015194 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] PATH_INFO: `/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b/users/1e245f7fc96942f08ab02f5714a134ab/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:46.017183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] RBAC: Authorizing `identity:create_grant(project_id=10ff00dd2cfb416f8a0ec01a679d0a6b, user_id=1e245f7fc96942f08ab02f5714a134ab, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:46.018659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb228d-fa9a-413f-ac0c-f43389633a31 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:46.036199 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 577/1153] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:19:45 2022] PUT /identity/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b/users/1e245f7fc96942f08ab02f5714a134ab/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:19:48.137956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491170-1aff-47f9-b099-71c74fa5c5e9 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:48.138835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491170-1aff-47f9-b099-71c74fa5c5e9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:48.138835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db491170-1aff-47f9-b099-71c74fa5c5e9 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:48.141015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 577/1154] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:48.147496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:48.147947 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:48.148051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:48.177775 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:48.179024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:48.260423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 578/1155] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:48.529630 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491170-1aff-47f9-b099-71c74fa5c5e9 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:48.644952 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db491170-1aff-47f9-b099-71c74fa5c5e9 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:48.664595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:48.681757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:48.682400 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:48.682664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:48.682931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:48.695112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:19:48.699885 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:48.701022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:19:48.702097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 578/1156] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:19:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:48.707311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:48.758653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:48.811646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e651e7c-c248-4b5f-ab22-6661e68f0468 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:48.848276 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:48.848873 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:48.849148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:48.849417 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:48.850108 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:48.850407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:48.851787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:48.860031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 579/1157] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:19:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:48.866565 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:48.971073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4d73f-900c-405d-a4a4-9be817c663b1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:49.007101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:49.007705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:49.007975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:49.008238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] PATH_INFO: `/v3/users/1e245f7fc96942f08ab02f5714a134ab` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:49.009377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] RBAC: Authorizing `identity:get_user(user_id=1e245f7fc96942f08ab02f5714a134ab)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:49.010833 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:49.012383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 579/1158] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:19:48 2022] GET /identity/v3/users/1e245f7fc96942f08ab02f5714a134ab => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:49.019017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:49.132848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb58c7-c0d7-42b2-a7ca-a6b6d768b0fc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:49.142997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:49.167887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:49.168487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:49.168742 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:49.169012 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:49.237049 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:49.238524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:49.340227 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 580/1159] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:49.349961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:49.408489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:49.460111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a6a38a-296b-484c-865b-3bf279a659b6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:49.494492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:49.495101 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:49.495370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:49.495629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] PATH_INFO: `/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:49.496769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] RBAC: Authorizing `identity:get_project(project_id=10ff00dd2cfb416f8a0ec01a679d0a6b)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:49.498217 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94257439-71e7-4df9-9519-afcae2be7d7b None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:49.499842 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 580/1160] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:19:49 2022] GET /identity/v3/projects/10ff00dd2cfb416f8a0ec01a679d0a6b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:49.507081 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:49.525043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:49.622215 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb5d69-d1da-434a-901e-0f83a90187ab None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:49.659662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:49.660258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:49.660531 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:49.660797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:49.661519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=10ff00dd2cfb416f8a0ec01a679d0a6b, user.id=1e245f7fc96942f08ab02f5714a134ab) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:49.661820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:49.663302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d811d-1205-42a1-bdcc-54d0f6dcd3d2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:49.669226 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 581/1161] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:19:49 2022] GET /identity/v3/role_assignments?user.id=1e245f7fc96942f08ab02f5714a134ab&scope.project.id=10ff00dd2cfb416f8a0ec01a679d0a6b&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:51.813490 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba01d-29e7-4430-a659-4d90d0e35ca3 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:51.814493 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba01d-29e7-4430-a659-4d90d0e35ca3 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:51.814493 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510ba01d-29e7-4430-a659-4d90d0e35ca3 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:51.815800 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 581/1162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:51.822278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:51.822998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:51.823150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:51.846032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:51.847242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:51.916520 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 582/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:52.177947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ba01d-29e7-4430-a659-4d90d0e35ca3 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:52.283472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510ba01d-29e7-4430-a659-4d90d0e35ca3 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:52.299522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:52.322096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:52.322772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:52.322977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:52.323238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:52.391183 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:52.392617 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:52.490414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 582/1164] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:19:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:19:52.497169 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:52.502143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:52.601966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f3788-e203-4efd-84cc-30281867108b None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:52.643751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:52.644674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:52.645030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:52.645361 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:52.653096 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:52.654589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-52ea293d-7d40-41e9-9c15-a64352cf610f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:19:52.666224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:19:52.667415 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:19:52.668622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 583/1165] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:19:52 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:19:52.673332 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:52.691334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:52.785866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95d6d40-c69d-4c33-a683-0e138949dd38 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:52.821370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:52.821979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:52.822249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:52.822514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:52.823149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:19:52.823449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:52.824789 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5000c8-e55b-484e-ba74-259327069695 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:52.831875 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 583/1166] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Sep 14 01:19:52 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:54.991231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2417-edc0-4c35-b706-75e5b490838f None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:54.991803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2417-edc0-4c35-b706-75e5b490838f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:54.991803 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494e2417-edc0-4c35-b706-75e5b490838f None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:54.993794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 584/1167] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:55.002734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65943-47bb-4cf1-bc36-d2559ba77d29 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:55.003125 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65943-47bb-4cf1-bc36-d2559ba77d29 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:55.003317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f65943-47bb-4cf1-bc36-d2559ba77d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:55.029247 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-45f65943-47bb-4cf1-bc36-d2559ba77d29 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:55.030473 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-45f65943-47bb-4cf1-bc36-d2559ba77d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:55.102638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 584/1168] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:55.379516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e2417-edc0-4c35-b706-75e5b490838f None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:55.466389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:55.505788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494e2417-edc0-4c35-b706-75e5b490838f None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:55.540932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:55.541523 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:55.541789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:55.542086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:55.542740 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:55.544118 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c24ff3-7ab1-4423-98f3-fda43e829b85 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:55.562180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 585/1169] 10.222.0.26 () {60 vars in 1138 bytes} [Wed Sep 14 01:19:55 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:19:57.695906 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499bcd-4e85-497e-9b60-0c03231c9ac8 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:57.696805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499bcd-4e85-497e-9b60-0c03231c9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:57.696805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3499bcd-4e85-497e-9b60-0c03231c9ac8 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:57.698960 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 585/1170] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:19:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:19:57.705283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:57.706039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:57.706339 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:57.732736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:19:57.733927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:19:57.803321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 586/1171] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:19:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:19:58.077835 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499bcd-4e85-497e-9b60-0c03231c9ac8 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:58.091551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:58.188131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3499bcd-4e85-497e-9b60-0c03231c9ac8 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:58.224031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:58.224686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:58.224959 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:58.225223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] PATH_INFO: `/v3/domains/62e021db13f64e8897b9fc4b8d1714b0` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:58.235678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=62e021db13f64e8897b9fc4b8d1714b0)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:58.237114 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0b0a6a-8db1-45f2-90a0-919a99451f94 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:58.238733 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 586/1172] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Sep 14 01:19:58 2022] GET /identity/v3/domains/62e021db13f64e8897b9fc4b8d1714b0 => generated 267 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:19:58.245316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:19:58.307770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:19:58.354244 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08154330-0547-4f64-9ced-01812d1f1122 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:19:58.389455 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:19:58.390045 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:19:58.390338 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:19:58.390603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:19:58.391247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:19:58.392659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e47209-88c1-46d5-9bcb-e822a575675d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:19:58.422458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 587/1173] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:19:58 2022] POST /identity/v3/projects => generated 361 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:20:00.552233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb27276-c710-48e3-a3f2-9fdc68f66823 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:00.552884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb27276-c710-48e3-a3f2-9fdc68f66823 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:00.552884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb27276-c710-48e3-a3f2-9fdc68f66823 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:00.554216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 587/1174] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:20:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:20:00.560576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:00.561287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:00.561420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:00.583090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:20:00.584280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:00.661491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 588/1175] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:20:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:20:00.924522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb27276-c710-48e3-a3f2-9fdc68f66823 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:01.001478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:01.035127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb27276-c710-48e3-a3f2-9fdc68f66823 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:01.071574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:01.072245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:01.072425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:01.072719 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] PATH_INFO: `/v3/domains/62e021db13f64e8897b9fc4b8d1714b0` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:01.074801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=62e021db13f64e8897b9fc4b8d1714b0)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:01.076239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c1d6aa-25c6-45d2-85ed-19403bdf6f4a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:01.077836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 588/1176] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Sep 14 01:20:00 2022] GET /identity/v3/domains/62e021db13f64e8897b9fc4b8d1714b0 => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:01.084521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:01.120424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:01.186295 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c5b39-1d76-47be-b57a-db9ce50affcd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:01.221503 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:01.222128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:01.222408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:01.222676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:01.223316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:01.224852 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799cf7ba-649f-4dfe-88cf-b52a8b2be68e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:01.249289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 589/1177] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:20:01 2022] POST /identity/v3/users => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:20:03.444558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71788-9723-4a48-8fe1-cfd4c315dc97 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:03.445446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71788-9723-4a48-8fe1-cfd4c315dc97 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:03.445446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a71788-9723-4a48-8fe1-cfd4c315dc97 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:03.447677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 589/1178] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:20:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:20:03.454156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:03.454602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:03.454696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:03.481304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:20:03.482538 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:03.552998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 590/1179] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:20:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:20:03.837030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a71788-9723-4a48-8fe1-cfd4c315dc97 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:03.947077 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a71788-9723-4a48-8fe1-cfd4c315dc97 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:03.985497 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:03.986146 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:03.986408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:03.986667 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:03.994214 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:03.999579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:20:04.004331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:20:04.005417 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:20:04.006696 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 590/1180] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:20:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:20:04.011811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:04.038180 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:04.118264 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8114d-bf47-4333-aa85-f713f6c96096 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:04.158342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:04.158963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:04.159230 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:04.159490 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:04.160153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:20:04.160445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:04.161816 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:04.171128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 591/1181] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:20:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:04.177464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:04.288301 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267276f-1322-44dc-ae03-0d59afdfe884 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:04.322521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:04.323173 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:04.323447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:04.323714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] PATH_INFO: `/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:04.337229 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b1c9cb3984eb58263ef62ca100dd9)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:04.338706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:04.340323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 591/1182] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:20:04 2022] GET /identity/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:04.349535 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:04.437410 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:04.456835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64b64a-6455-4e1a-8a2d-90d66b678bea None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:04.493196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:04.493786 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:04.494086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:04.494352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:04.563548 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:04.565321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:04.651563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:04.670967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 592/1183] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:20:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:20:04.678066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:04.735551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:04.784468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed60dbcd-cd7e-43e1-8239-0bb2d07e0785 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:04.820692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:04.821360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:04.821708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:04.821972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] PATH_INFO: `/v3/projects/06ea6fa8405f44a3bed050578bce345e` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:04.823152 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] RBAC: Authorizing `identity:get_project(project_id=06ea6fa8405f44a3bed050578bce345e)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:04.824579 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570fd57a-d9e9-494d-bf63-3a6520166f84 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:04.826209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 592/1184] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:20:04 2022] GET /identity/v3/projects/06ea6fa8405f44a3bed050578bce345e => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:04.832437 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:04.937885 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80470c93-fdf0-49a9-84cb-1a8747ac1f28 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:04.978789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:04.979375 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:04.979644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:04.979905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:04.980621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=06ea6fa8405f44a3bed050578bce345e, user.id=0f4b1c9cb3984eb58263ef62ca100dd9) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:20:04.980927 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:04.985505 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172edebe-2f11-4ffa-be52-2a18b19838fc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:04.991327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 593/1185] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:20:04 2022] GET /identity/v3/role_assignments?user.id=0f4b1c9cb3984eb58263ef62ca100dd9&scope.project.id=06ea6fa8405f44a3bed050578bce345e&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:07.084534 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e47d-5261-44a1-94fc-5e3c038eb104 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:07.084975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e47d-5261-44a1-94fc-5e3c038eb104 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:07.085085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79e47d-5261-44a1-94fc-5e3c038eb104 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:07.086481 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 593/1186] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:20:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:20:07.091707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:07.092158 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:07.092258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:07.111578 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:20:07.112763 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:07.156800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:07.181438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 594/1187] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:20:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:20:07.457857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e47d-5261-44a1-94fc-5e3c038eb104 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:07.563687 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79e47d-5261-44a1-94fc-5e3c038eb104 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:07.588096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:07.601202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:07.601827 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:07.602132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:07.602390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:07.613895 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:20:07.618765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:20:07.619829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:20:07.621027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 594/1188] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:20:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:20:07.625892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:07.722767 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f6756a-4209-4526-a14c-3757aea548fc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:07.757458 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:07.758039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:07.758342 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:07.758602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:07.759261 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:20:07.759553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:07.760912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:07.769790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 595/1189] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:20:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:07.776691 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:07.884804 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46893145-704a-48e2-b3a4-1bf475907505 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:07.919994 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:07.920603 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:07.920872 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:07.921134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] PATH_INFO: `/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:07.922410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b1c9cb3984eb58263ef62ca100dd9)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:07.923837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:07.925388 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 595/1190] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:20:07 2022] GET /identity/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:07.931334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:07.975633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:08.034586 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac05c9f9-df8e-48bf-829f-bdd5816e5540 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:08.080257 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:08.080887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:08.081157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:08.081421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] PATH_INFO: `/v3/projects/06ea6fa8405f44a3bed050578bce345e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:08.082615 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] RBAC: Authorizing `identity:get_project(project_id=06ea6fa8405f44a3bed050578bce345e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:08.084091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9922f72-42f3-49f4-9498-36772dcdcc96 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:08.085729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 596/1191] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:20:07 2022] GET /identity/v3/projects/06ea6fa8405f44a3bed050578bce345e => generated 361 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:08.095676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:08.149829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:08.198889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c2cf33-cc67-4164-997e-87dbbf95b4df None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:08.233572 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:08.234185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:08.234460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:08.234721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] PATH_INFO: `/v3/projects/06ea6fa8405f44a3bed050578bce345e/users/0f4b1c9cb3984eb58263ef62ca100dd9/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:08.236854 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] RBAC: Authorizing `identity:create_grant(project_id=06ea6fa8405f44a3bed050578bce345e, user_id=0f4b1c9cb3984eb58263ef62ca100dd9, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:08.238315 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f504a16-4965-43e2-afea-19c39e14e0df None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:08.257453 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 596/1192] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:20:08 2022] PUT /identity/v3/projects/06ea6fa8405f44a3bed050578bce345e/users/0f4b1c9cb3984eb58263ef62ca100dd9/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:20:10.384027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287e21-1ee4-41fb-838f-1ecb8b7faff8 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:10.384571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287e21-1ee4-41fb-838f-1ecb8b7faff8 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:10.384571 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98287e21-1ee4-41fb-838f-1ecb8b7faff8 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:10.385915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 597/1193] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:20:10.392423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:10.393144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:10.393283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:10.418988 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:20:10.420169 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:20:10.497092 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 597/1194] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:20:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:20:10.765279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98287e21-1ee4-41fb-838f-1ecb8b7faff8 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:10.885791 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98287e21-1ee4-41fb-838f-1ecb8b7faff8 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:10.894716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:10.920612 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:10.921222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:10.921491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:10.921755 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:10.932621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:20:10.937383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:20:10.938472 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:20:10.940316 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 598/1195] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:20:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:20:10.946912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:10.991994 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:11.052181 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdae21b-d0f7-4607-af82-5be8d6e96187 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:11.088283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:11.088893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:11.089167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:11.089434 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:11.090122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:20:11.090422 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:11.091797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:11.099793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 598/1196] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:20:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:11.106089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:11.208980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406dd2c-b3a4-4de9-9b35-c17c0eca568a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:11.243441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:11.244042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:11.244312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:11.244573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] PATH_INFO: `/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:11.245725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] RBAC: Authorizing `identity:get_user(user_id=0f4b1c9cb3984eb58263ef62ca100dd9)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:11.247186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:11.248729 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 599/1197] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Sep 14 01:20:11 2022] GET /identity/v3/users/0f4b1c9cb3984eb58263ef62ca100dd9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:11.257360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:11.373252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f862a3-c8bd-4eb4-944d-fce69d2b4dec None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:11.375872 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:11.411475 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:11.412090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:11.412379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:11.412645 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:11.481395 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:11.482850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:11.592673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 599/1198] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:20:11 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:20:11.598969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:11.638317 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:11.705524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f85a2-6afe-48bb-bc6e-5559b02fd3b4 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:11.740909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:11.741529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:11.741799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:11.742088 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] PATH_INFO: `/v3/projects/06ea6fa8405f44a3bed050578bce345e` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:11.743409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] RBAC: Authorizing `identity:get_project(project_id=06ea6fa8405f44a3bed050578bce345e)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:11.744860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c33db4-b3e9-4bc7-aa2d-b93a377c206b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:11.746519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 600/1199] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Sep 14 01:20:11 2022] GET /identity/v3/projects/06ea6fa8405f44a3bed050578bce345e => generated 361 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:11.753289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:20:11.757318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:11.854435 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ba15d4-f8f2-4068-8239-a9c443eaef45 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:20:11.897489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:20:11.898351 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:20:11.898769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:20:11.899174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:20:11.900316 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=06ea6fa8405f44a3bed050578bce345e, user.id=0f4b1c9cb3984eb58263ef62ca100dd9) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:20:11.900760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:20:11.903083 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c00d7-2385-4bb7-a1e3-9ff9faa906fd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:20:11.912438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 600/1200] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:20:11 2022] GET /identity/v3/role_assignments?user.id=0f4b1c9cb3984eb58263ef62ca100dd9&scope.project.id=06ea6fa8405f44a3bed050578bce345e&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:20:21.735345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=48 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:20:21.908102 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=66 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:07.378738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a96d3-3564-418a-944e-cb2d3e10b566 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:07.378738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a96d3-3564-418a-944e-cb2d3e10b566 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:07.378738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a96d3-3564-418a-944e-cb2d3e10b566 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:07.378738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 601/1201] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:07.386419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:07.387018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:07.387018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:07.413064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:07.414618 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:07.494578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 601/1202] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:07.813546 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a96d3-3564-418a-944e-cb2d3e10b566 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:07.951728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a96d3-3564-418a-944e-cb2d3e10b566 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:07.992788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:07.993932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:07.994240 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:07.994533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:08.009416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:08.009416 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:08.025212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:08.026376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:08.027942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:08.027942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:08.027942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:08.027942 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 602/1203] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:08.033158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:08.167974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247ac632-3061-4727-8357-2a33853a12e7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:08.201196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:08.208275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:08.208962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:08.209467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:08.209752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:08.210489 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:08.210819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:08.212509 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537e31b7-cbfc-426a-9c9a-73b69e09d4c6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:08.225773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 602/1204] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:21:08.231902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:08.297570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:08.365475 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ca85d-c947-43d9-881f-d1597a7cf843 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:08.410156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:08.410860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:08.411141 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:08.411337 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:08.412017 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:08.413542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d29a8-dc2c-4b04-a4d8-f68c8ccb1a88 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:08.445566 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 603/1205] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Sep 14 01:21:08 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 14 01:21:10.763468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e4db-8451-4b2c-b9a4-2648015bc4e7 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:10.764071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e4db-8451-4b2c-b9a4-2648015bc4e7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:10.764071 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32e4db-8451-4b2c-b9a4-2648015bc4e7 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:10.765350 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 603/1206] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:21:10.771198 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:10.771643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:10.771836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:10.793031 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:10.794278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:10.871124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 604/1207] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:11.138857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32e4db-8451-4b2c-b9a4-2648015bc4e7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:11.254175 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32e4db-8451-4b2c-b9a4-2648015bc4e7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:11.288487 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:11.290174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:11.290174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:11.290174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:11.301404 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:11.306769 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:11.307764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:11.309066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:11.316783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 604/1208] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:21:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:11.317817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:11.360098 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:11.424207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeef766-5388-432e-8c33-0d4dc8874f59 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:11.470273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:11.471964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:11.471964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:11.471964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:11.472232 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:11.472461 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:11.473954 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:11.485136 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 605/1209] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:21:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:11.492723 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:11.552132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:11.605252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a3fa51-69ef-43d0-beef-29d1e5c0dada None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:11.644838 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:11.644838 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:11.644838 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:11.645234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:11.657035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:11.658580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:11.673817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:11.675409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:11.676586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:11.676586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:11.676586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:11.676586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 605/1210] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:11.681413 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:11.800076 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:11.803938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1091800-52d5-474c-bf2f-9984593ba96d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:11.852829 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:11.853913 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:11.854248 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:11.854518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:11.855163 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:11.855631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:11.856811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:11.870147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 606/1211] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:11.876343 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:11.986147 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bc8ac-263e-484c-9eaf-3a7688dff5d5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:11.990187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:12.024904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:12.025595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:12.025864 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:12.026163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:12.034936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:12.036444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:12.048846 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:12.050186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:12.051165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 606/1212] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:12.056253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:12.174647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d29e171-e0fe-4811-819a-cc9fd8e08e01 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:12.210623 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:12.211325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:12.211600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:12.211874 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:12.212684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:12.212987 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:12.214542 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:12.233944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 607/1213] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:21:12.240130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:12.347364 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3edf328-7c99-4572-868f-bf2020e20a7f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:12.386015 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:12.386845 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:12.387243 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:12.387642 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:12.423264 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:12.470395 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:12.471993 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:12.576163 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 607/1214] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:21:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:21:12.582333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:12.604204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:12.690311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebfda90-2265-4513-bab1-6b7811fa6c05 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:12.728258 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:12.729080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:12.729360 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:12.729716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:12.741881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:12.743452 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:12.759848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:12.761093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:12.766734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:12.766734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:12.766734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:12.766734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 608/1215] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:12.766979 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:12.832132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:12.872338 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94c8c1-e6e8-4700-a9b3-2d97750ca2ef None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:12.907294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:12.907973 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:12.908299 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:12.908626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:12.909317 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:12.909677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:12.911123 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:12.927112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 608/1216] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:12.928527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:13.003800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:13.039651 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0188cee1-a2f2-40a1-88de-352e1e815078 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:13.086900 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:13.087579 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:13.087961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:13.088326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:13.098353 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:13.099865 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:13.111948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:13.113116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:13.114236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:13.115853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 609/1217] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:13.121628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:13.242474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57407c1-7213-41ef-90ef-503bad18df95 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:13.269577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:13.285965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:13.286852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:13.287137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:13.287560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:13.288529 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:13.288844 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:13.290656 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c169c782-1fdf-4d27-a27f-386b8d2d4eed None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:13.306760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 609/1218] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:13.313097 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:13.425554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff7705f-463d-4fb0-9d4b-d682e44e2a63 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:13.466314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:13.474100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:13.474713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:13.475020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:13.475289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:13.476011 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=ce6a85b7c0c14f76a471680d63078065) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:13.476312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:13.477759 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63eaefe-7c55-43eb-b6bd-34fbe9e9bef2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:13.495391 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 610/1219] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:21:13 2022] GET /identity/v3/role_assignments?user.id=ce6a85b7c0c14f76a471680d63078065&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:15.658687 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9627c-4e91-46b8-8714-b1844819c024 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:15.659259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9627c-4e91-46b8-8714-b1844819c024 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:15.659259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9627c-4e91-46b8-8714-b1844819c024 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:15.660626 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 610/1220] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:15.666464 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:15.667051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:15.667051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:15.696427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:15.697898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:15.785355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 611/1221] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:16.081255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9627c-4e91-46b8-8714-b1844819c024 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:16.196660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9627c-4e91-46b8-8714-b1844819c024 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:16.230912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:16.231544 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:16.231829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:16.232078 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:16.242611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:16.247688 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:16.248757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:16.249615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 611/1222] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:16.255443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:16.356804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e311a5-9ed7-4259-80ab-57fdfa620f4d None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:16.380608 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:16.391858 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:16.392482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:16.392733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:16.392990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:16.393650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] RBAC: Adding query filter params (name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:16.393946 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:16.395362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:16.403986 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 612/1223] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:16.410574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:16.465729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:16.530705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d48c9fc-fad3-4716-81ba-6a4e7890bd5b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:16.567611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:16.568392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:16.568739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:16.569030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:16.579850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:16.581459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:16.596686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:16.603249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:16.604465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:16.604465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:16.604465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:16.604465 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 612/1224] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:16.604781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:16.711563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8a0ed-fbe6-4df3-9418-def80b3663da None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:16.748783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:16.749446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:16.749728 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:16.750129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:16.750813 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:16.751183 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:16.752781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:16.768210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 613/1225] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:16.775036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:16.875545 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:16.882998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19240b-12dc-4d51-b4c0-33d9c8a17fd4 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:16.917313 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:16.917969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:16.918276 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:16.918542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:16.926005 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:16.927455 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:16.938865 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:16.940491 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:16.941620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 613/1226] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:16.946688 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.001225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:17.061471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d742635-3ac4-4ca4-8da2-89592fc36ffc None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:17.105999 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:17.106624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:17.106899 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:17.107168 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:17.107911 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:17.108217 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:17.109684 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:17.128675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 614/1227] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:17.135212 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.248394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2b206-fb3a-463c-8f6d-896be276d00a None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:17.286330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:17.286966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:17.287356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:17.287704 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:17.302126 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:17.303969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:17.320492 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:17.321664 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:17.322739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:17.322739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:17.322739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:17.322739 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 614/1228] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:17.330341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.444408 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abc5bdc-fbc0-42ed-9da8-e82e5f0a272f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:17.447255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:17.485821 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:17.486484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:17.486877 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:17.487155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:17.487859 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:17.488187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:17.489659 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:17.502547 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 615/1229] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:17.508635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.521495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:17.622239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c706f36-cc36-4c1d-b7d6-4f9e5fa64438 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:17.656787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:17.657419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:17.657692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:17.657965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:17.668165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:17.669615 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:17.682595 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:17.683661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:17.684775 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 615/1230] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:17.690537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.814790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cc2a0-cf0d-4510-8c22-c946a90fdfff None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:17.854881 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:17.855550 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:17.856001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:17.856321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:17.857246 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:17.857602 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:17.859279 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1471e1-2402-4503-8016-63ca9d0d30b2 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:17.874379 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 616/1231] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:17.880686 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:17.948741 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:17.983580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07849ebb-65e0-451f-8aea-ac628abe7ac6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:18.018912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:18.019736 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] REQUEST_METHOD: `PUT` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:18.019978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:18.020258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/ce6a85b7c0c14f76a471680d63078065/roles/571b656acd2c4f66a4e5c4525e297e01` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:18.035658 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=ce6a85b7c0c14f76a471680d63078065, role_id=571b656acd2c4f66a4e5c4525e297e01)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:18.037259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec21278-2403-46f7-8491-57d223009211 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:18.050594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 616/1232] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:21:17 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/ce6a85b7c0c14f76a471680d63078065/roles/571b656acd2c4f66a4e5c4525e297e01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 14 01:21:20.204457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759cb16-6825-440b-99a1-cd321be41399 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:20.205054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759cb16-6825-440b-99a1-cd321be41399 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:20.205054 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d759cb16-6825-440b-99a1-cd321be41399 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:20.210018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 617/1233] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:20.212408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:20.212934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:20.213099 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:20.233200 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:20.234618 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:20.306902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 617/1234] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:21:20.579797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759cb16-6825-440b-99a1-cd321be41399 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:20.613517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:20.697074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d759cb16-6825-440b-99a1-cd321be41399 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:20.732857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:20.733494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:20.733769 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:20.734082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:20.745176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:20.750042 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:20.751592 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:20.752487 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 618/1235] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Sep 14 01:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:20.757685 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:20.809693 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:20.869057 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79fbab2-d057-4246-829b-b44a861fc7ee None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:20.915471 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:20.916085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:20.916353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:20.916614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:20.917282 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] RBAC: Adding query filter params (name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:20.917573 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:20.918975 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:20.927386 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 618/1236] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Sep 14 01:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:21:20.934624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:21.037402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:21.055789 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881ff949-3cb3-4e00-99ea-81a31898b4f5 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:21.097909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:21.098610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:21.098882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:21.099146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:21.112811 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:21.114377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:21.129764 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:21.131009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:21.132204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:21.132204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:21.132204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:21.132204 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 619/1237] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:21.137166 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:21.248734 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f7b3a3-2b54-4b10-89aa-4ca970c71659 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:21.259252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:21.294628 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:21.297857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:21.297857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:21.297857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:21.297857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:21.297857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:21.298559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:21.314562 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 619/1238] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:21.320573 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:21.443836 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c81e00f-8041-4f96-8f8f-75e8b7b37876 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:21.481539 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:21.482570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:21.483053 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:21.483524 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:21.488326 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:21.495765 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:21.498143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:21.512356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:21.513407 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:21.515208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:21.520211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 620/1239] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:21.521285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:21.644120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef622e1-6849-4249-bae1-90c119127c87 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:21.691377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:21.692021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:21.692300 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:21.692568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:21.693304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:21.693722 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:21.695394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:21.715246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 620/1240] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:21.721208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:21.843799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8562707-41b3-41d2-9dca-4c9ebdbdabad None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:21.885778 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:21.886474 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:21.886819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:21.887065 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:21.963720 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:21.970564 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:21.972051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.082972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 621/1241] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:21:21 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:21:22.089378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:22.123984 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:22.203144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611c374d-14bf-49ff-9a7b-3cb2062cd4b6 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:22.235611 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:22.236207 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:22.236484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:22.236745 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:22.247418 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:22.249308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:22.264989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:22.266482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:22.267964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:22.267964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:22.267964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:22.267964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 621/1242] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:22.272174 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:22.373968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:22.380471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e1400d-308c-4276-b720-e2bdc7ef4c4f None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:22.417801 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:22.418419 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:22.418697 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:22.419026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:22.419745 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:22.420043 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:22.421641 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.434645 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 622/1243] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:21:22.441018 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:22.525937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:22.549121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95db36a-9485-43f2-9d24-1f9f1e244fb5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:22.585786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:22.586583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:22.586841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:22.587128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:22.595103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:22.596661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:22.608773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:22.609877 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:22.610989 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 622/1244] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:22.616162 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:22.734247 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b8aa4e-7465-41a2-9712-6b1d7289c309 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:22.775374 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:22.776260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:22.776600 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:22.776879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:22.777966 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:22.778304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:22.780249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ffe9aa-665a-457c-be0f-d8c09eb78991 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.797636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 623/1245] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:22.804934 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:22.913612 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b0f19c-8059-42a7-8461-00ae72fe028e None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:22.950673 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:22.951283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:22.951555 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:22.951819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:22.952568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] RBAC: Adding query filter params (role.id=571b656acd2c4f66a4e5c4525e297e01, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=ce6a85b7c0c14f76a471680d63078065) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:22.952871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:22.954342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54525dd-48ef-4bf0-b3ec-bc20a3135d97 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:22.960392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 623/1246] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:21:22 2022] GET /identity/v3/role_assignments?user.id=ce6a85b7c0c14f76a471680d63078065&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=571b656acd2c4f66a4e5c4525e297e01 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:25.274521 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9825b5-4e7c-4007-bd19-e32a9d3455d1 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:25.275075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9825b5-4e7c-4007-bd19-e32a9d3455d1 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:25.275209 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9825b5-4e7c-4007-bd19-e32a9d3455d1 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:25.276517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 624/1247] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:25.282570 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:25.283055 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:25.283149 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:25.290156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:25.304160 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:25.305737 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:25.385676 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 624/1248] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:25.692477 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9825b5-4e7c-4007-bd19-e32a9d3455d1 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:25.713343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:25.822936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9825b5-4e7c-4007-bd19-e32a9d3455d1 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:25.858814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:25.859449 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:25.859725 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:25.860113 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:25.871892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:25.877454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:25.878917 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:25.880142 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:25.880860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 625/1249] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:21:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:25.887285 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:25.999298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf96f6fd-3cdc-4878-80a3-df12a0c68485 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.038714 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:26.045477 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.046277 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.046627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.046897 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:26.047561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:26.047860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:26.049251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:26.058948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 625/1250] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:21:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:26.065898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:26.165434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:26.193766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f62cfd7-0873-4dcd-985c-0f0629e4d785 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.230076 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.230657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.230931 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.231205 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:26.241574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:26.243079 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:26.259496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:26.260460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:26.261348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:26.261348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:26.261348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:26.261348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 626/1251] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:26.266576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:26.384931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96109c54-6a0b-4613-bcc7-cd38f7719fdf None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.425823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.426470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.426746 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.427012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:26.427655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:26.428068 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:26.429449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:26.442143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 626/1252] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:26.448026 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:26.563537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b53209-46d3-4863-8eeb-cde678cfbf7d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.592814 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:26.601325 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.601958 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.602274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.602543 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:26.610533 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:26.612071 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:26.623610 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:26.624575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:26.625526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 627/1253] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:26.630840 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:26.691424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:26.740177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e0f95-d2f8-4a62-8621-bd73b20040a7 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.790232 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.791291 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.791729 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.792199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:26.793321 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:26.793816 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:26.796333 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:26.818755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 627/1254] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:26.824790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:26.933114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c8b0b-5418-4650-b5b9-bbd23406a484 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:26.968197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:26.968804 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:26.969077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:26.969344 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:27.024299 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:27.038541 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:27.040187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:27.151074 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 628/1255] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:21:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:21:27.157195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:27.262323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cfe421c-2adc-4732-96ad-d7c4584831eb None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:27.276876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:27.299896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:27.300537 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:27.300813 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:27.301079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:27.311765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:27.313216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:27.330818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:27.331851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:27.332841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:27.332841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:27.332841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:27.332841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 628/1256] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:27.338450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:27.447187 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:27.459955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c5fa02-578b-46cc-a821-5fe8d28a4a22 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:27.500736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:27.501352 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:27.501626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:27.501936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:27.502817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:27.503245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:27.504807 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:27.522116 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 629/1257] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:27.523454 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:27.643639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d1ec-d3e6-4301-9546-7399f80fc8d5 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:27.681604 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:27.682311 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:27.682587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:27.682851 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:27.690947 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:27.692469 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:27.704837 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:27.706064 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:27.707037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 629/1258] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:27.717124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:27.845015 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b42de0-5e7f-412b-86c1-55fb44ad0c86 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:27.879413 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:27.880022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:27.880388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:27.880747 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] PATH_INFO: `/v3/projects` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:27.881509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:27.881812 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:27.883304 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ea3dc0-dccc-4c3a-bd21-25444cd7d79b None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:27.897694 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 630/1259] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:27.904151 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:27.933377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:28.021585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb1d7db-7274-4595-96ce-00b63c183f05 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:28.059194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:28.059805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:28.060073 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:28.060336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:28.061040 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=ce6a85b7c0c14f76a471680d63078065) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:28.061337 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:28.062798 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb876ac-465b-40cf-9ea9-c9649c7e0cd8 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:28.072258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 630/1260] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:21:27 2022] GET /identity/v3/role_assignments?user.id=ce6a85b7c0c14f76a471680d63078065&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 14 01:21:30.191263 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011688a-3980-4295-b149-8aac4ec7f346 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:30.192166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011688a-3980-4295-b149-8aac4ec7f346 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:30.192166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011688a-3980-4295-b149-8aac4ec7f346 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:30.194324 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 631/1261] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:30.199683 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:30.200097 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:30.200185 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:30.226750 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:30.228629 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:30.305602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 631/1262] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:30.568041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011688a-3980-4295-b149-8aac4ec7f346 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:30.586632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:30.690332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4011688a-3980-4295-b149-8aac4ec7f346 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:30.726682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:30.727288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:30.727554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:30.727819 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:30.738721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:30.743574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:30.744879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:30.745749 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 632/1263] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:21:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:30.750747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:30.773533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:30.859424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1546932e-0ad1-452a-8402-f2cc3e5a172b None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:30.894509 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:30.895112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:30.895381 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:30.895647 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] PATH_INFO: `/v3/roles` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:30.896304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:30.896599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:30.898020 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:30.906246 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 632/1264] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:21:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:30.914519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:30.992963 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:31.040022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc58ee-3b0a-4def-b1af-083d9d73847a None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:31.090210 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:31.090857 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:31.091129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:31.091396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:31.102627 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:31.104085 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:31.119372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:31.120434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:31.121333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:31.121333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:31.121333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:31.121333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 633/1265] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:31.128841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:31.226463 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:31.247135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def1a0e-5c6a-461a-8f78-af75424e377f None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:31.283133 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:31.283727 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:31.284003 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:31.284269 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:31.284909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:31.285207 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:31.286610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:31.299830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 633/1266] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:31.306665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:31.413516 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe16b1-7c06-467f-a583-6ce0343dbc56 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:31.433051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:31.450271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:31.450898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:31.451175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:31.451441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] PATH_INFO: `/v3/users/placement` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:31.459100 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:31.460580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:31.472155 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:31.473252 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:31.474431 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 634/1267] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:31.480025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:31.591449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79ce36-7c70-42fe-8247-64a32f4e38b1 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:31.626259 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:31.626849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:31.627120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:31.627389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] PATH_INFO: `/v3/users` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:31.628105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:31.628406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:31.629831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:31.638765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:31.647632 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 634/1268] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:31.653909 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:31.781468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f1c968-7adf-431f-b6e3-5fe9afc786fd None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:31.834761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:31.835460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:31.835618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:31.835905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:31.846843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:31.848275 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:31.862951 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:31.864010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:31.865214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:31.865214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:31.865214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:31.865214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 635/1269] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:31.870890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:31.987590 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263a4f08-cc13-4537-a11c-ebaef8f74cea None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:32.023539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:32.024132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:32.024403 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:32.024665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:32.025294 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:32.025586 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:32.026996 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:32.039681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 635/1270] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:32.045611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:32.085737 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:32.156069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbce78-8925-42e2-88e6-066a5c59ae30 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:32.190623 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:32.191239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:32.191512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:32.191780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:32.198629 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:32.200086 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:32.211383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:32.212389 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:32.213440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 636/1271] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:32.218484 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:32.276298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:32.322806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87433c8f-af5c-49d8-9b87-9cc7b57a1f01 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:32.356855 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:32.357438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:32.357703 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:32.357944 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:32.358650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:32.358943 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:32.360347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e42a3b-9b8f-4d10-aad7-ac169f15ce62 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:32.372006 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 636/1272] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:32.378710 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:32.486624 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:32.496596 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9faaea-4227-4871-836e-f31e2a386010 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:32.531835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:32.532479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] REQUEST_METHOD: `PUT` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:32.532750 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:32.533022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] PATH_INFO: `/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/ce6a85b7c0c14f76a471680d63078065/roles/43f052e6109f426bb045d2114f5e67f8` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:32.535278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] RBAC: Authorizing `identity:create_grant(project_id=c9f08639310641c5a5070068ee6f1c76, user_id=ce6a85b7c0c14f76a471680d63078065, role_id=43f052e6109f426bb045d2114f5e67f8)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:32.536715 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3b0142-4c92-4be8-bb18-19f4ab5e21bd None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:32.552249 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 637/1273] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Sep 14 01:21:32 2022] PUT /identity/v3/projects/c9f08639310641c5a5070068ee6f1c76/users/ce6a85b7c0c14f76a471680d63078065/roles/43f052e6109f426bb045d2114f5e67f8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 14 01:21:34.680557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48050d-37ff-4684-9e25-ddfd39f80b07 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:34.681143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48050d-37ff-4684-9e25-ddfd39f80b07 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:34.681143 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48050d-37ff-4684-9e25-ddfd39f80b07 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:34.683253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 637/1274] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:34.692442 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:34.692837 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:34.693011 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:34.718640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:34.719845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:34.800554 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 638/1275] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:35.080777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48050d-37ff-4684-9e25-ddfd39f80b07 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.199132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48050d-37ff-4684-9e25-ddfd39f80b07 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:35.219784 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:35.239761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:35.240375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:35.240670 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:35.240932 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:35.252253 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 14 01:21:35.257305 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:35.258408 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:35.259824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 638/1276] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:35.268465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.361396 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:35.382650 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715c6154-8f63-434d-8881-c4f9674777d9 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:35.418029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:35.418635 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:35.418912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:35.419176 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] PATH_INFO: `/v3/roles` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:35.419842 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:35.420143 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:35.421522 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:35.429717 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 639/1277] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:35.435787 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.535870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dd4104-f2f1-4500-be36-77845428390b None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:35.570916 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:35.571523 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:35.571792 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:35.572052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:35.582236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:35.583677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:35.598019 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:35.599130 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:35.600378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:35.600378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:35.600378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:35.600378 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 639/1278] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:35.605203 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.712892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd999cb-9039-480b-92e8-26f49a88d496 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:35.750965 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:35.751794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:35.752101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:35.752392 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] PATH_INFO: `/v3/domains` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:35.753179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:35.753633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:35.755574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:35.769206 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 640/1279] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:35.775090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.807067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:35.880999 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52b882-9fde-4876-bb10-6f2dcb2d4212 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:35.916131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:35.916743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:35.917012 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:35.917275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:35.924699 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:35.926168 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 14 01:21:35.943889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:35.944978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:35.946142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 640/1280] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:35.951519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:35.957288 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:36.055409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899f0b30-e317-4feb-a77d-5007200ce804 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:36.091970 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:36.092569 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:36.092843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:36.093102 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] PATH_INFO: `/v3/users` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:36.093808 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:36.094132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:36.095555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:36.112815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 641/1281] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Sep 14 01:21:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:36.118954 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:36.207702 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:36.218698 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e32c93-ee82-4201-bb78-82e9f01f95ad None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:36.254540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:36.255174 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:36.255447 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:36.255710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:36.324602 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:36.326022 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:36.424341 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:36.434755 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 641/1282] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Sep 14 01:21:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 14 01:21:36.441253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:36.506479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:36.546753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5664e5df-17bf-45e2-9f5b-267a8ddc686d None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:36.583955 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:36.584526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:36.584790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:36.585052 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:36.596355 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:36.597798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 14 01:21:36.613146 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:36.614150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 14 01:21:36.615150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 14 01:21:36.615150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 14 01:21:36.615150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:36.615150 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 642/1283] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Sep 14 01:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 14 01:21:36.620467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:36.724879 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9762bd-d846-49af-a53a-68088210f6bd None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:36.764376 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:36.764972 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:36.765249 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:36.765514 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:36.766177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:36.766472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:36.767858 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:36.779075 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 642/1284] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Sep 14 01:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:36.785022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:36.893314 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c8182-9805-4e2d-8e5d-4dab63560b1e None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:36.910771 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:36.933213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:36.933845 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:36.934148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:36.934418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:36.941509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:36.942980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 14 01:21:36.954960 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:36.955990 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 14 01:21:36.956935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: ERROR keystone.server.flask.application Sep 14 01:21:36.958152 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 643/1285] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Sep 14 01:21:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:36.962911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:37.035344 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:37.088008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b344f595-d54b-4b3f-b0f9-195eb84137a2 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:37.125830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:37.126438 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:37.126709 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:37.126977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] PATH_INFO: `/v3/projects` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:37.127659 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:37.127962 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:37.129382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8993ac6d-557b-4e0f-86fa-2a5de508cccd None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:37.141205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 643/1286] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Sep 14 01:21:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:37.149805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:37.256517 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57b12d9-0713-4143-9a34-c5e7cac96c17 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:37.297112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:37.297709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:37.298013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:37.298271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:37.299004 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] RBAC: Adding query filter params (role.id=43f052e6109f426bb045d2114f5e67f8, scope.project.id=c9f08639310641c5a5070068ee6f1c76, user.id=ce6a85b7c0c14f76a471680d63078065) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:37.299311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:37.300758 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0382c01f-d580-4e96-b894-941a73616679 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:37.306678 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 644/1287] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Sep 14 01:21:37 2022] GET /identity/v3/role_assignments?user.id=ce6a85b7c0c14f76a471680d63078065&scope.project.id=c9f08639310641c5a5070068ee6f1c76&role.id=43f052e6109f426bb045d2114f5e67f8 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:39.435675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fa12-a12a-431b-abc7-caf939a44ebf None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:39.436583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fa12-a12a-431b-abc7-caf939a44ebf None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:39.436583 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898fa12-a12a-431b-abc7-caf939a44ebf None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:39.438766 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 644/1288] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:39.444582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:39.445001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:39.445091 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:39.464805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:39.465980 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:39.496231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:39.534649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 645/1289] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:39.797708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898fa12-a12a-431b-abc7-caf939a44ebf None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:39.907742 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898fa12-a12a-431b-abc7-caf939a44ebf None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:39.918218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:39.944345 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:39.944966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:39.945237 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:39.945505 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:39.946140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:39.947502 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:39.956379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:39.957641 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:39.958771 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 645/1290] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:21:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:39.964095 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:40.074211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0e4985-6536-4e38-a9c1-0fd658caa29e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:40.108797 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:40.109402 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:40.109672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:40.109939 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:40.110595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:40.110889 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:40.112271 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33035a4-9264-4b75-8e9d-9d80ca9f6b83 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:40.116935 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 646/1291] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 14 01:21:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:40.125974 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:40.231805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ae58f-e046-49bc-abf5-58ca3e6b6253 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:40.266188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:40.266764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:40.267032 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:40.267296 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] PATH_INFO: `/v3/services` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:40.267912 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:40.268199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:40.269553 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b4b528-c9fa-4c3c-8642-4ea98e5ff3b5 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:40.274115 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 646/1292] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 14 01:21:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:42.379618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cef8-7fcf-4f88-8fcb-db3bd6681bbe None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:42.380484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cef8-7fcf-4f88-8fcb-db3bd6681bbe None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:42.380484 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1041cef8-7fcf-4f88-8fcb-db3bd6681bbe None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:42.382666 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 647/1293] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:42.391421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359aad8-5d5e-475b-9b38-412460dc956e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:42.391836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359aad8-5d5e-475b-9b38-412460dc956e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:42.392002 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a359aad8-5d5e-475b-9b38-412460dc956e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:42.412850 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-a359aad8-5d5e-475b-9b38-412460dc956e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:42.414065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a359aad8-5d5e-475b-9b38-412460dc956e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:42.447994 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:42.488536 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 647/1294] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:42.753450 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041cef8-7fcf-4f88-8fcb-db3bd6681bbe None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:42.790280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:42.864480 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1041cef8-7fcf-4f88-8fcb-db3bd6681bbe None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:42.899078 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:42.899672 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:42.899945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:42.900208 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:42.900864 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:42.902278 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1ce8b2-0303-4785-8ea1-410a4ed00ac1 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:42.920075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 648/1295] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Sep 14 01:21:42 2022] POST /identity/v3/services => generated 246 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:21:45.029930 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9752c-cbad-48da-84a6-95a8d72db729 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:45.030825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9752c-cbad-48da-84a6-95a8d72db729 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:45.031027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c9752c-cbad-48da-84a6-95a8d72db729 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:45.033194 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 648/1296] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:45.040724 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:45.041192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:45.041286 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:45.060077 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:45.061245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:45.137378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 649/1297] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 14 01:21:45.399577 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9752c-cbad-48da-84a6-95a8d72db729 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:45.515421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c9752c-cbad-48da-84a6-95a8d72db729 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:45.551421 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:45.552027 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:45.552293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:45.552554 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:45.553190 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:45.554588 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:45.563674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:45.565137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:45.566621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 649/1298] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:21:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:45.573023 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:45.601855 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:45.688083 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42047e5c-377f-4c91-9803-e8de1a160abc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:45.723329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:45.723920 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:45.724192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:45.724453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:45.725128 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:45.725425 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:45.726820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:45.732321 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 650/1299] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 14 01:21:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:45.739066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:45.781075 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:45.859302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3793d045-ddec-4d8a-bc59-080465afbf9d None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:45.892236 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:45.892834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:45.893104 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:45.893367 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:45.894119 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] RBAC: Adding query filter params (interface=public, service_id=1a94bd0d97bb4b8582b6a91f2294369f, region_id=RegionOne) {{(pid=81414) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:45.894419 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:45.895779 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f60f98d-81bf-4904-a749-e5015ea862d7 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:45.900818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 650/1300] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Sep 14 01:21:45 2022] GET /identity/v3/endpoints?service_id=1a94bd0d97bb4b8582b6a91f2294369f&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:45.906696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:45.993867 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:46.012467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4050ca8-97fd-436a-bbbe-8335de340064 None admin] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:46.050611 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:46.051225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:46.051468 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:46.051733 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:46.052336 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:46.053781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc67cf8-aa91-4bc8-9c4b-71bb9322592a None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:46.061133 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 651/1301] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Sep 14 01:21:45 2022] GET /identity/v3/services => generated 2191 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 14 01:21:48.144025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e8c6-47f4-4883-96b6-950ba8f10be8 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:48.144831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e8c6-47f4-4883-96b6-950ba8f10be8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:48.144831 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e8c6-47f4-4883-96b6-950ba8f10be8 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:48.147251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 651/1302] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:21:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:21:48.153448 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:48.153895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:48.154020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:48.181703 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:21:48.182922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:21:48.251800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 652/1303] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:21:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 14 01:21:48.528275 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84e8c6-47f4-4883-96b6-950ba8f10be8 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:48.635627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:48.644024 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a84e8c6-47f4-4883-96b6-950ba8f10be8 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:48.681884 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:48.682539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:48.682810 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:48.683076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:48.683654 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:48.685014 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 14 01:21:48.694323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 14 01:21:48.695533 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.get_or_create( Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application with Lock( Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self._enter() Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application return self.creator() Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application created_value = creator( Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: ERROR keystone.server.flask.application Sep 14 01:21:48.696420 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 652/1304] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Sep 14 01:21:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 14 01:21:48.702421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:48.791492 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:48.815671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5888d1-eebe-4281-95a9-b0ec55e14302 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:48.850673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:48.851287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:48.851553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:48.851818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] PATH_INFO: `/v3/services` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:48.852440 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81415) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 14 01:21:48.852738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:48.854137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa9a28-b2a0-443c-a3ad-07c0defea316 None admin] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:48.859333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 653/1305] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Sep 14 01:21:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 14 01:21:48.868560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:21:48.981151 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf083cc-1290-4168-a192-7c7641cc68a9 None admin] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:21:49.015713 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f48f44237b4c4aa48da7b17eda7c3281', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:21:49.016289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:21:49.016557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:21:49.016819 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:21:49.017474 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:21:49.018860 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76f604-42e1-4416-9aec-0af72e34eb6c None admin] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:21:49.041904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 653/1306] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Sep 14 01:21:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 14 01:21:58.856665 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=30 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:21:59.037139 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:52.588915 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0169-baf3-4562-96c1-90ac84e78a57 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:52.589434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0169-baf3-4562-96c1-90ac84e78a57 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:52.589434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0169-baf3-4562-96c1-90ac84e78a57 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:52.591392 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 654/1307] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:22:52.601035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5978-53a2-4812-bb7f-7f35faad0932 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:52.602485 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5978-53a2-4812-bb7f-7f35faad0932 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:52.603478 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef5978-53a2-4812-bb7f-7f35faad0932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:52.768234 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-4cef5978-53a2-4812-bb7f-7f35faad0932 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:22:52.770363 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4cef5978-53a2-4812-bb7f-7f35faad0932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:52.904576 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 654/1308] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:22:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:22:53.195432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:53.195843 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:53.196041 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:53.197365 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 655/1309] 10.222.0.26 () {54 vars in 965 bytes} [Wed Sep 14 01:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:22:53.203528 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4848-415e-41ab-81d5-b92bf37d9e30 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:53.203808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4848-415e-41ab-81d5-b92bf37d9e30 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:53.204078 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd4848-415e-41ab-81d5-b92bf37d9e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:53.288690 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-acbd4848-415e-41ab-81d5-b92bf37d9e30 None None] MFA Rules not processed for user `1f5db3c037944dfbb2e4dd1bd8fe24b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:22:53.290346 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-acbd4848-415e-41ab-81d5-b92bf37d9e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:53.315574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:53.380128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 655/1310] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Sep 14 01:22:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:22:53.389555 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:53.391388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:53.520323 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c28268-0e0b-4e6f-a86a-c8207564bb95 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:53.601234 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:53.601967 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:53.602290 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:53.602482 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:53.604200 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:53.670817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:53.731300 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:53.732738 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e23aa6-753f-4e93-8155-db1e66b36ae0 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:53.751853 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 656/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:22:56.176308 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:56.177165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:56.177165 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:56.179257 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 656/1312] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:22:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:22:56.184817 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f167d8-3c23-4910-96fa-db7838b2b7d7 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:56.185265 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f167d8-3c23-4910-96fa-db7838b2b7d7 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:56.185356 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f167d8-3c23-4910-96fa-db7838b2b7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:56.264492 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-46f167d8-3c23-4910-96fa-db7838b2b7d7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:22:56.265622 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-46f167d8-3c23-4910-96fa-db7838b2b7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:56.369273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 657/1313] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:22:56.661705 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:56.663966 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:56.783482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57811219-4ac2-46e5-82ea-68e014c7576e None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:22:56.858752 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:56.861578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:22:56.862344 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:56.862880 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:56.863161 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:56.865318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:56.984205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:22:56.985743 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b86cbd8-297e-4857-81e6-b29b41f3d040 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:22:57.005594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 657/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:22:59.457101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:59.458101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:59.458101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:59.460160 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 658/1315] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:22:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:22:59.465825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1111d-dd9b-4be8-81a6-594755e22a61 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:22:59.466270 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1111d-dd9b-4be8-81a6-594755e22a61 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:22:59.466362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d1111d-dd9b-4be8-81a6-594755e22a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:22:59.535000 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-25d1111d-dd9b-4be8-81a6-594755e22a61 None None] MFA Rules not processed for user `3341ca87c68d41e8a973c93cb5345109`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:22:59.536198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-25d1111d-dd9b-4be8-81a6-594755e22a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:59.588429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:22:59.634873 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 658/1316] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:22:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:22:59.928570 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:22:59.931101 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:22:59.946736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:00.059591 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadec714-c12d-482e-bc02-538da20e4a74 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:00.136800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:00.137409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:00.137682 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:00.137945 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:00.139582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:00.204658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:00.252548 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:00.254009 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d053260-0f2d-42cb-90ff-94db1e593717 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:00.272868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 659/1317] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:22:59 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:02.911805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:02.912701 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:02.912701 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:02.915048 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 659/1318] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:02.920551 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f08f1-cbb0-4051-97ae-5a58c904c35d None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:02.920986 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f08f1-cbb0-4051-97ae-5a58c904c35d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:02.921089 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f08f1-cbb0-4051-97ae-5a58c904c35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:02.990362 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-fe4f08f1-cbb0-4051-97ae-5a58c904c35d None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:02.991581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fe4f08f1-cbb0-4051-97ae-5a58c904c35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:03.091512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 660/1319] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:03.370585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:03.372140 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:03.484437 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b45bb-8ddc-4729-bfd0-1041c9b2f4e6 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:03.561398 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:03.562010 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:03.562310 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:03.562582 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:03.564177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:03.571829 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:03.696117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:03.697509 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15148d1b-1fd1-45f8-b6b8-9ca787e1b08b service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:03.717089 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 660/1320] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:06.490496 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f31a2-fef9-4e75-a1c2-8994e1a6b594 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:06.491044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f31a2-fef9-4e75-a1c2-8994e1a6b594 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:06.491044 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f31a2-fef9-4e75-a1c2-8994e1a6b594 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:06.492417 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 661/1321] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:06.497861 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb37b6-50a4-4c77-9994-e7d60b99c570 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:06.498350 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb37b6-50a4-4c77-9994-e7d60b99c570 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:06.498440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddb37b6-50a4-4c77-9994-e7d60b99c570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:06.570188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-3ddb37b6-50a4-4c77-9994-e7d60b99c570 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:06.571454 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-3ddb37b6-50a4-4c77-9994-e7d60b99c570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:06.636117 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:06.679866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 661/1322] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:06.974713 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:06.975103 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:06.975441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:06.976890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 662/1323] 10.222.0.26 () {54 vars in 965 bytes} [Wed Sep 14 01:23:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:06.983087 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21cd1-8a24-4f1c-83d3-adb0ac5a0667 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:06.983446 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21cd1-8a24-4f1c-83d3-adb0ac5a0667 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:06.983783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf21cd1-8a24-4f1c-83d3-adb0ac5a0667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:07.065457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-abf21cd1-8a24-4f1c-83d3-adb0ac5a0667 None None] MFA Rules not processed for user `1f5db3c037944dfbb2e4dd1bd8fe24b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:07.066692 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-abf21cd1-8a24-4f1c-83d3-adb0ac5a0667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:07.147946 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 662/1324] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Sep 14 01:23:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:07.157148 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:07.158723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:07.179030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:07.264014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b93d5e4-4512-4a87-800a-a736e20bfabd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:07.338904 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:07.339513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:07.339783 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:07.340051 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:07.341643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:07.415752 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:07.456005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:07.457605 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f8402-2cf0-4823-a75c-730f754df3be service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:07.475895 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 663/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:09.889874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:09.889874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:09.890501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:09.891596 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 663/1326] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:09.897195 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf0df-8cc5-4a2b-ac63-88f27b0abb76 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:09.897640 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf0df-8cc5-4a2b-ac63-88f27b0abb76 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:09.897751 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ecf0df-8cc5-4a2b-ac63-88f27b0abb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:09.965024 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-45ecf0df-8cc5-4a2b-ac63-88f27b0abb76 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:09.966269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-45ecf0df-8cc5-4a2b-ac63-88f27b0abb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:10.064545 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 664/1327] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:10.338030 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:10.339622 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:10.404801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:10.449627 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5e652-fc55-4f07-b62a-6bb9430d01b1 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:10.523781 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:10.524432 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:10.524710 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:10.524978 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:10.526650 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:10.634963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:10.637208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:10.638711 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a179b1c1-a81f-4cc3-9fa4-ef7cacb790e6 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:10.657862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 664/1328] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:12.850719 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:12.851197 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:12.851312 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:12.852662 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 665/1329] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:12.858698 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc7b5-1f6a-40f3-950b-64f9ea427a12 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:12.859266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc7b5-1f6a-40f3-950b-64f9ea427a12 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:12.859266 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bc7b5-1f6a-40f3-950b-64f9ea427a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:12.933011 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-cf9bc7b5-1f6a-40f3-950b-64f9ea427a12 None None] MFA Rules not processed for user `3341ca87c68d41e8a973c93cb5345109`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:12.934340 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-cf9bc7b5-1f6a-40f3-950b-64f9ea427a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:13.040142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 665/1330] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:13.316285 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:13.317860 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:13.351465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:13.427941 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cd9dec-bf42-40a8-a9ed-8ef141f35c55 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:13.509235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:13.510018 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:13.510115 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:13.510366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:13.511983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:13.598159 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:13.631020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:13.632447 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02de0e4d-ff68-4ec9-9241-94aa14ef2015 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:13.651798 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 666/1331] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:13 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:16.019121 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:16.019674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:16.019674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:16.021105 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 666/1332] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:16.027462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b720c0-8427-45f9-99b3-adab781020ca None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:16.027904 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b720c0-8427-45f9-99b3-adab781020ca None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:16.028016 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b720c0-8427-45f9-99b3-adab781020ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:16.107974 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-96b720c0-8427-45f9-99b3-adab781020ca None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:16.109191 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-96b720c0-8427-45f9-99b3-adab781020ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:16.211655 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 667/1333] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:16.489892 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:16.492268 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:16.578223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:16.615801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fea665-5f33-4ff2-8084-5c0b07417ca9 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:16.693961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:16.694610 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:16.694887 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:16.695155 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:16.696778 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:16.811406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:16.812765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c15b7d-4357-418d-be09-7032bbc58bae service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:16.823242 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:16.831796 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 667/1334] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:19.004093 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:19.004979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:19.004979 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:19.007198 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 668/1335] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:19.013587 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a036e6-567f-4f63-b4dd-051611311c5e None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:19.014342 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a036e6-567f-4f63-b4dd-051611311c5e None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:19.014526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a036e6-567f-4f63-b4dd-051611311c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:19.092988 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-40a036e6-567f-4f63-b4dd-051611311c5e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:19.094279 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-40a036e6-567f-4f63-b4dd-051611311c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:19.194413 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 668/1336] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:19.479882 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:19.481438 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:19.526676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:19.595357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd59465-dbec-4093-98dd-823112bf0b56 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:19.671030 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:19.671626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:19.671892 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:19.672157 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:19.673721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:19.759173 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:19.786747 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:19.788181 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba565f3e-2503-4ecc-9340-97ec482a5176 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:19.806562 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 669/1337] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:22.263245 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:22.263795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:22.263795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:22.265076 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 669/1338] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:22.271553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bd987-d0f8-43c6-9d67-f89130f0ed22 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:22.272393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bd987-d0f8-43c6-9d67-f89130f0ed22 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:22.272526 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08bd987-d0f8-43c6-9d67-f89130f0ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:22.359067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-d08bd987-d0f8-43c6-9d67-f89130f0ed22 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:22.360274 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-d08bd987-d0f8-43c6-9d67-f89130f0ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:22.462799 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 670/1339] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:22.739389 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:22.740933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:22.824786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:22.852059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec5c6a1-d983-4fd1-aab0-876373f0064d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:22.936920 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:22.937543 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:22.937811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:22.938109 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:22.939725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:23.052786 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:23.054167 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48161f0-0f7b-4301-9a6d-f9fd90538103 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:23.076193 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 670/1340] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:27.532241 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:27.532868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:27.532868 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:27.534387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 671/1341] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:27.539621 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e776-e301-44d3-bee9-e282c0e976a0 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:27.540070 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e776-e301-44d3-bee9-e282c0e976a0 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:27.540172 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3319e776-e301-44d3-bee9-e282c0e976a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:27.545795 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:27.608241 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-3319e776-e301-44d3-bee9-e282c0e976a0 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:27.609361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-3319e776-e301-44d3-bee9-e282c0e976a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:27.715289 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 671/1342] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:27.994787 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:27.996289 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:28.043214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:28.105129 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94b30d7-eda8-44cc-85e4-958da7e3abb9 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:28.181639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:28.182284 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:28.182557 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:28.182822 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:28.184429 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:28.279796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:28.296420 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:28.297886 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81135989-5282-4f43-9086-67d97a5dd015 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:28.316308 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 672/1343] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:31.049905 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:31.050457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:31.050457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:31.051762 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 672/1344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:31.057282 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0c7f6-f1cf-461f-9b3e-b897c5ecfbad None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:31.057696 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0c7f6-f1cf-461f-9b3e-b897c5ecfbad None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:31.057964 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc0c7f6-f1cf-461f-9b3e-b897c5ecfbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:31.132046 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-0bc0c7f6-f1cf-461f-9b3e-b897c5ecfbad None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:31.133254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0bc0c7f6-f1cf-461f-9b3e-b897c5ecfbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:31.230898 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 673/1345] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:31.508666 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:31.510360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:31.603890 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:31.622990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc766e6a-7843-481e-ba0b-5d14ab0d9b78 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:31.705038 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:31.705652 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:31.705928 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:31.706228 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:31.707822 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:31.833843 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:31.835233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f58c76-8189-43cf-b65e-71ed11b7e691 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:31.853708 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 673/1346] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:36.228998 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:36.229519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:36.229519 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:36.231401 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 674/1347] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:36.238120 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109db4fb-da78-4d67-8acb-a69fbc24515c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:36.238540 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109db4fb-da78-4d67-8acb-a69fbc24515c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:36.238651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109db4fb-da78-4d67-8acb-a69fbc24515c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:36.256862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:36.314955 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-109db4fb-da78-4d67-8acb-a69fbc24515c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:36.316535 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-109db4fb-da78-4d67-8acb-a69fbc24515c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:36.417208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 674/1348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:36.702029 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:36.704443 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:36.771633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:36.819260 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb36b18f-11fe-4940-8257-bdac710a0dbf None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:36.894630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:36.895236 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:36.895510 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:36.895776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:36.897385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:37.018075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:37.022428 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:37.024609 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47ba7c5-4d27-495d-82d2-8dd863e6deef service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:37.043847 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 675/1349] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:41.934375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceab4-10ef-41e0-b02f-ff7b50fe0c38 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:41.934841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceab4-10ef-41e0-b02f-ff7b50fe0c38 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:41.934936 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eceab4-10ef-41e0-b02f-ff7b50fe0c38 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:41.936551 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 675/1350] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 14 01:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:41.942137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546169ea-3386-49a1-8578-0ae23b23caca None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:41.942539 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546169ea-3386-49a1-8578-0ae23b23caca None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:41.942628 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546169ea-3386-49a1-8578-0ae23b23caca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:42.053388 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-546169ea-3386-49a1-8578-0ae23b23caca None None] MFA Rules not processed for user `ce6a85b7c0c14f76a471680d63078065`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:42.055340 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-546169ea-3386-49a1-8578-0ae23b23caca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:42.199020 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 676/1351] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 14 01:23:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:46.183718 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d37c-be29-4c28-880b-882d2d2fbb02 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:46.184273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d37c-be29-4c28-880b-882d2d2fbb02 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:46.184273 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11d37c-be29-4c28-880b-882d2d2fbb02 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:46.185594 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 676/1352] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 14 01:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:46.191069 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23ae10-571a-4a82-9edd-5f20cdb0097d None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:46.191483 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23ae10-571a-4a82-9edd-5f20cdb0097d None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:46.191626 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb23ae10-571a-4a82-9edd-5f20cdb0097d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:46.249189 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:46.277457 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-fb23ae10-571a-4a82-9edd-5f20cdb0097d None None] MFA Rules not processed for user `ce6a85b7c0c14f76a471680d63078065`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:46.278884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fb23ae10-571a-4a82-9edd-5f20cdb0097d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:46.394156 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 677/1353] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 14 01:23:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:46.415319 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=59 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:47.802507 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fea2f5-ec6f-47bd-893f-a6aad7497d11 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:47.803195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fea2f5-ec6f-47bd-893f-a6aad7497d11 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:47.803195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fea2f5-ec6f-47bd-893f-a6aad7497d11 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:47.804400 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 677/1354] 10.222.0.26 () {54 vars in 921 bytes} [Wed Sep 14 01:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:47.809896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b0d34-70dd-4d25-b514-1c47ee4846b6 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:47.810380 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b0d34-70dd-4d25-b514-1c47ee4846b6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:47.810491 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1b0d34-70dd-4d25-b514-1c47ee4846b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:47.889212 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-de1b0d34-70dd-4d25-b514-1c47ee4846b6 None None] MFA Rules not processed for user `ce6a85b7c0c14f76a471680d63078065`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:47.890479 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-de1b0d34-70dd-4d25-b514-1c47ee4846b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:47.932753 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:47.999145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 678/1355] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Sep 14 01:23:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:51.542112 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbd0db-3cb9-4391-9b7c-6ce0b173480c None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:51.542812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbd0db-3cb9-4391-9b7c-6ce0b173480c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:51.542812 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cbd0db-3cb9-4391-9b7c-6ce0b173480c None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:51.544031 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 678/1356] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:23:51.549434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25dd3-a4b9-4ed7-b433-8266a17ce716 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:51.549902 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25dd3-a4b9-4ed7-b433-8266a17ce716 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:51.550022 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d25dd3-a4b9-4ed7-b433-8266a17ce716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:51.622504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-a8d25dd3-a4b9-4ed7-b433-8266a17ce716 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:51.623701 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a8d25dd3-a4b9-4ed7-b433-8266a17ce716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:51.730135 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 679/1357] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:52.022634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:52.022841 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:52.023283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:52.025110 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 679/1358] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 14 01:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:52.037310 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37837605-19ac-45ed-993d-c17c72997660 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:52.038130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37837605-19ac-45ed-993d-c17c72997660 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:52.038648 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37837605-19ac-45ed-993d-c17c72997660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:52.058815 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:52.128575 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-37837605-19ac-45ed-993d-c17c72997660 None None] MFA Rules not processed for user `6f6158bf6948476a9244d0c28e1c8404`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:52.130027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-37837605-19ac-45ed-993d-c17c72997660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:52.261716 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 680/1359] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 14 01:23:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:52.270158 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:52.271815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:52.429299 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3517a15-c164-4fe0-b0d8-a3908365660c None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:52.503964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:52.544808 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:52.545600 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:52.545969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:52.546360 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:52.548037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:52.648467 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:52.650283 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358a23e9-4698-4a17-b141-e50844602ce8 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:52.668818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 680/1360] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:55.035002 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:55.035561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:55.035561 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:55.037139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 681/1361] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:55.042883 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e6e7-e7ec-4fdb-9de1-60d793d19a84 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:55.043424 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e6e7-e7ec-4fdb-9de1-60d793d19a84 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:55.043620 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a4e6e7-e7ec-4fdb-9de1-60d793d19a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:55.139202 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b1a4e6e7-e7ec-4fdb-9de1-60d793d19a84 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:55.140445 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b1a4e6e7-e7ec-4fdb-9de1-60d793d19a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:55.182655 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:55.244785 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 681/1362] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:55.535233 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:55.536754 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:55.610056 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:55.710348 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a52942d-2b8b-470a-a60a-f066c42093ba None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:55.839446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:55.840060 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:55.840328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:55.841363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:55.843032 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:55.867175 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:55.936671 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:55.938359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccebd4c-8196-4593-83fe-4086476f6bcd service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:55.957494 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 682/1363] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:58.427830 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87806a9-6aa0-445d-b728-6f1c2be86435 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:58.428441 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87806a9-6aa0-445d-b728-6f1c2be86435 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:58.428441 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87806a9-6aa0-445d-b728-6f1c2be86435 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:58.429963 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 682/1364] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:58.435647 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eef6ef-baf6-4341-95c0-abc01ae0ac2e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:58.436066 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eef6ef-baf6-4341-95c0-abc01ae0ac2e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:58.436173 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eef6ef-baf6-4341-95c0-abc01ae0ac2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:58.505280 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-b2eef6ef-baf6-4341-95c0-abc01ae0ac2e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:58.506529 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b2eef6ef-baf6-4341-95c0-abc01ae0ac2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:58.608285 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:58.617794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 683/1365] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:23:58.906034 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:58.906580 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:58.906945 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:58.908457 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 683/1366] 10.222.0.26 () {54 vars in 962 bytes} [Wed Sep 14 01:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:58.914114 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541112-e093-4c8c-ad04-367ed20db139 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:58.914377 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541112-e093-4c8c-ad04-367ed20db139 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:58.914660 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d541112-e093-4c8c-ad04-367ed20db139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:58.993938 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-7d541112-e093-4c8c-ad04-367ed20db139 None None] MFA Rules not processed for user `6f6158bf6948476a9244d0c28e1c8404`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:58.995171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-7d541112-e093-4c8c-ad04-367ed20db139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:59.129385 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 684/1367] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Sep 14 01:23:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:23:59.137687 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:23:59.139599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:59.295953 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4613cdf-1569-450e-bcb1-04b93864593d None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:23:59.317488 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:59.415524 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:23:59.416128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:59.416399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:59.416662 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:59.418357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:59.524859 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:23:59.527782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b569ba3-9fa8-4bf9-a19f-55337a59395c service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:23:59.550727 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 684/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:23:59.632423 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2148eb8-d13d-4d2a-9d86-c853c74160a3 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:59.632968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2148eb8-d13d-4d2a-9d86-c853c74160a3 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:59.632968 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2148eb8-d13d-4d2a-9d86-c853c74160a3 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:59.634298 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 685/1369] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:23:59.640041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea41b-9c55-44a5-b50e-7ff1fe83d21c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:23:59.640327 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea41b-9c55-44a5-b50e-7ff1fe83d21c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:23:59.640598 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fea41b-9c55-44a5-b50e-7ff1fe83d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:23:59.682352 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:23:59.727430 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d5fea41b-9c55-44a5-b50e-7ff1fe83d21c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:23:59.728677 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d5fea41b-9c55-44a5-b50e-7ff1fe83d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:23:59.849462 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 685/1370] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:00.175076 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:00.175425 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:00.175890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:00.177631 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 686/1371] 10.222.0.26 () {54 vars in 964 bytes} [Wed Sep 14 01:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:00.182834 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445ea30-badc-46d4-ac62-8030dffcc092 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:00.183218 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445ea30-badc-46d4-ac62-8030dffcc092 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:00.183407 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5445ea30-badc-46d4-ac62-8030dffcc092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:00.264672 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-5445ea30-badc-46d4-ac62-8030dffcc092 None None] MFA Rules not processed for user `cb571f4363c34886b8763cab9310459f`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:00.265901 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-5445ea30-badc-46d4-ac62-8030dffcc092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:00.317910 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:00.349406 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 686/1372] 10.222.0.26 () {58 vars in 1061 bytes} [Wed Sep 14 01:24:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:00.357793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:00.359346 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:00.412560 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:00.472319 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21de0fe-52de-45b5-9f79-de9e813a42cd None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:00.561658 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cb571f4363c34886b8763cab9310459f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:00.562589 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:00.563179 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:00.563676 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:00.565435 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:00.711061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:00.725818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:00.727395 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dcdac7-011f-458d-84b9-cc93d778635f service cinder] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:00.747653 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 687/1373] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:01.793354 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:01.793891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:01.793891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:01.795184 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 687/1374] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:01.800601 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f952a-4842-43f6-934b-4eeac47ad164 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:01.801094 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f952a-4842-43f6-934b-4eeac47ad164 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:01.801222 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f952a-4842-43f6-934b-4eeac47ad164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:01.871687 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-375f952a-4842-43f6-934b-4eeac47ad164 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:01.872901 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-375f952a-4842-43f6-934b-4eeac47ad164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:01.972132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 688/1375] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:02.256402 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:02.257954 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:02.417805 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8954b-6f61-46f0-b297-00600fc5dd13 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:02.470969 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:02.534717 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:02.535330 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:02.535592 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:02.535853 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:02.537454 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:02.635651 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:02.637306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1377c0-b839-45a8-9e0a-91a8a4ce78c5 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:02.656498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 688/1376] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:04.863997 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:04.864541 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:04.864541 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:04.865850 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 689/1377] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:04.871647 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49b10-c189-45ca-ba7a-9fd83a0aa1f2 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:04.872137 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49b10-c189-45ca-ba7a-9fd83a0aa1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:04.872379 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d49b10-c189-45ca-ba7a-9fd83a0aa1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:04.939614 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d1d49b10-c189-45ca-ba7a-9fd83a0aa1f2 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:04.941298 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d1d49b10-c189-45ca-ba7a-9fd83a0aa1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:04.948941 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:05.061799 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 689/1378] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:05.344636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:05.346196 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:05.421681 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:05.513774 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327230b-d3f8-4216-a8b5-e675d0a0de45 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:05.633306 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:05.633932 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:05.634432 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:05.634706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:05.636441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:05.656376 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:05.734273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:05.736618 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34259ecd-48f1-4e13-afe1-618e736d2ad4 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:05.758372 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 690/1379] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:07.951329 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:07.951857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:07.951857 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:07.953409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 690/1380] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:07.958780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50173c-f622-4318-9eac-89e25af04906 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:07.959214 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50173c-f622-4318-9eac-89e25af04906 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:07.959328 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50173c-f622-4318-9eac-89e25af04906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:08.052537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-4f50173c-f622-4318-9eac-89e25af04906 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:08.053848 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4f50173c-f622-4318-9eac-89e25af04906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:08.169629 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:08.176912 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 691/1381] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:08.460889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:08.462558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:08.608135 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:08.619889 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a142e28-3f07-4f6b-a41a-03d5c8f9e939 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:08.753362 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:08.754086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:08.754599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:08.754869 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:08.756656 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:08.888665 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:08.890368 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf3ffd-5283-486f-bdad-2db3f8ce065e service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:08.900599 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:08.914132 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 691/1382] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 14 01:24:11.130905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:11.131700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:11.131700 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:11.133061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 692/1383] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:24:11.138904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78da94-1aee-406e-8b27-87e13a9d6cba None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:11.139377 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78da94-1aee-406e-8b27-87e13a9d6cba None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:11.139475 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78da94-1aee-406e-8b27-87e13a9d6cba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:11.233783 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-6e78da94-1aee-406e-8b27-87e13a9d6cba None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:11.235797 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-6e78da94-1aee-406e-8b27-87e13a9d6cba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:11.387209 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 692/1384] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:11.686890 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:11.688574 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:11.888075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b80a5d-5652-4397-af42-ae4e6b9f80dc None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:11.975548 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:12.041434 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:12.042124 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:12.042527 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:12.042825 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:12.044430 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:12.142643 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:12.144341 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e55992-4623-4997-b146-95d5a6070e85 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:12.166239 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 693/1385] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:14.375757 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:14.376356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:14.376356 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:14.377825 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 693/1386] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:14.383139 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be927d-8b76-4bc9-997f-3027f6d08e28 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:14.383581 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be927d-8b76-4bc9-997f-3027f6d08e28 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:14.383677 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19be927d-8b76-4bc9-997f-3027f6d08e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:14.453251 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-19be927d-8b76-4bc9-997f-3027f6d08e28 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:14.454520 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-19be927d-8b76-4bc9-997f-3027f6d08e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:14.487722 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:14.552644 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 694/1387] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:14.839095 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:14.840678 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:14.922661 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:14.997177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb78db24-741c-44fc-be41-e8db50fcde3a None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:15.111915 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:15.112531 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:15.112800 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:15.113067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:15.114674 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:15.153177 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:15.205904 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:15.207539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4f5f-35c3-4326-8104-8cc1faa8ec06 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:15.227390 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 694/1388] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:15.831025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f9654-19e2-4ac9-8608-45d8a43cb345 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:15.831667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f9654-19e2-4ac9-8608-45d8a43cb345 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:15.831667 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f9654-19e2-4ac9-8608-45d8a43cb345 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:15.832905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 695/1389] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:15.838226 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94a257-52ae-4c32-98a7-eda4fc156aa7 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:15.838675 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94a257-52ae-4c32-98a7-eda4fc156aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:15.838793 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d94a257-52ae-4c32-98a7-eda4fc156aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:15.909530 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-2d94a257-52ae-4c32-98a7-eda4fc156aa7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:15.910782 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-2d94a257-52ae-4c32-98a7-eda4fc156aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:16.013444 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 695/1390] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:16.305699 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58181-5273-4817-9eb4-056b4115116a None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:16.306010 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58181-5273-4817-9eb4-056b4115116a None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:16.306283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d58181-5273-4817-9eb4-056b4115116a None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:16.307637 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 696/1391] 10.222.0.26 () {54 vars in 966 bytes} [Wed Sep 14 01:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:16.313199 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d560b-8e61-4191-958b-9ab3cad482e8 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:16.313486 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d560b-8e61-4191-958b-9ab3cad482e8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:16.313754 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d560b-8e61-4191-958b-9ab3cad482e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:16.331238 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:16.394503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-7c6d560b-8e61-4191-958b-9ab3cad482e8 None None] MFA Rules not processed for user `0aad071f42b547938aac8b6063e4a1bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:16.395707 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7c6d560b-8e61-4191-958b-9ab3cad482e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:16.477909 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 696/1392] 10.222.0.26 () {58 vars in 1063 bytes} [Wed Sep 14 01:24:16 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:16.487378 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d58181-5273-4817-9eb4-056b4115116a None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:16.489359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-86d58181-5273-4817-9eb4-056b4115116a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:16.597250 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d58181-5273-4817-9eb4-056b4115116a None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:16.657480 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:16.674887 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0aad071f42b547938aac8b6063e4a1bd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:16.675504 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:16.675782 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:16.676050 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:16.677673 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:16.792293 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:16.793735 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b50733-1ac3-410d-8a62-abc32bf5db24 service glance] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:16.812580 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 697/1393] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:17.372387 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78429-e24d-4fa6-a9a9-de3fb879e70b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:17.372933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78429-e24d-4fa6-a9a9-de3fb879e70b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:17.372933 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d78429-e24d-4fa6-a9a9-de3fb879e70b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:17.374429 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 697/1394] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:17.380269 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f53b-21a1-4721-b8f1-5188c4fb51b3 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:17.380820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f53b-21a1-4721-b8f1-5188c4fb51b3 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:17.380820 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0f53b-21a1-4721-b8f1-5188c4fb51b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:17.458451 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-49a0f53b-21a1-4721-b8f1-5188c4fb51b3 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:17.459692 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-49a0f53b-21a1-4721-b8f1-5188c4fb51b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:17.503359 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:17.569818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 698/1395] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:17.577466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999254-7f96-414c-87a5-cae4afce7189 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:17.577821 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999254-7f96-414c-87a5-cae4afce7189 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:17.578188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92999254-7f96-414c-87a5-cae4afce7189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:17.647134 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-92999254-7f96-414c-87a5-cae4afce7189 None None] MFA Rules not processed for user `cf3b078d5a134266a19c889b477fb60f`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:17.648353 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-92999254-7f96-414c-87a5-cae4afce7189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:17.707988 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:17.740501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 698/1396] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Sep 14 01:24:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:17.786679 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:17.786957 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:17.787221 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:17.788582 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 699/1397] 10.222.0.26 () {54 vars in 943 bytes} [Wed Sep 14 01:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:17.794929 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:17.795281 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:17.795639 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:17.859981 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:17.861585 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:17.873037 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] MFA Rules not processed for user `03da2d917c354a389dda2242ac5d4297`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:17.874336 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:17.979836 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 699/1398] 10.222.0.26 () {58 vars in 1040 bytes} [Wed Sep 14 01:24:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:24:17.989372 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:17.998568 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3ff56-6141-4fe5-8c92-91c657623b30 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:18.012471 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5383ac54-4e16-4087-a0c5-27c5b581d2d2 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:18.025470 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:18.038826 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:18.105142 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '03da2d917c354a389dda2242ac5d4297', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:18.105681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:18.105956 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:18.106251 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:18.107876 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:18.135373 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:18.135983 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:18.136255 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:18.136584 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:18.138224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:18.193575 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:18.195460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ab96de-2ba3-46db-843a-6a77d7e41acd service swift] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:18.203561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 700/1399] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Sep 14 01:24:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 14 01:24:18.233652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:18.235291 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce48e1b-a06c-4d7c-abc3-bc1d4eeb089a service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:18.253879 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 700/1400] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:20.452198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:20.452820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:20.452820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:20.454131 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 701/1401] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:20.459367 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa8d8-458f-4a56-9c08-1b4ef3e75064 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:20.459794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa8d8-458f-4a56-9c08-1b4ef3e75064 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:20.459908 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffaaa8d8-458f-4a56-9c08-1b4ef3e75064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:20.499453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:20.531888 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-ffaaa8d8-458f-4a56-9c08-1b4ef3e75064 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:20.533242 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-ffaaa8d8-458f-4a56-9c08-1b4ef3e75064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:20.653563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 701/1402] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:20.955765 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:20.957324 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:21.027306 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:21.126862 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fb03ee-948f-4c46-8742-36289bd592d7 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:21.249738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:21.250368 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:21.250634 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:21.250896 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:21.252503 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:21.274186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:21.354178 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:21.356059 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfeed3-d6e5-4e8d-a15e-393cc7170f76 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:21.375522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 702/1403] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:23.606766 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:23.607329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:23.607427 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:23.609025 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 702/1404] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:23.614852 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407f7e4-460b-42ef-91db-b0163be2ffff None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:23.615383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407f7e4-460b-42ef-91db-b0163be2ffff None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:23.615383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407f7e4-460b-42ef-91db-b0163be2ffff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:23.690067 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-4407f7e4-460b-42ef-91db-b0163be2ffff None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:23.691732 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4407f7e4-460b-42ef-91db-b0163be2ffff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:23.797721 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:23.806427 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 703/1405] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:24.100393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:24.102134 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:24.218727 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:24.261936 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164c584-bad4-4e65-9ad0-c6e42569f946 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:24.398454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:24.398454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:24.398454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:24.398454 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:24.399001 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:24.487301 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:24.508163 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:24.510035 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ec20f-5cff-4de2-8928-493ea3a4813e service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:24.528706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 703/1406] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:26.733870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:26.733870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:26.733870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:26.733870 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 704/1407] 10.222.0.26 () {54 vars in 927 bytes} [Wed Sep 14 01:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:26.738132 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1974-422d-45c0-b801-ec1891c3a855 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:26.738597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1974-422d-45c0-b801-ec1891c3a855 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:26.738707 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c1974-422d-45c0-b801-ec1891c3a855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:26.815202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-522c1974-422d-45c0-b801-ec1891c3a855 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:26.816793 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-522c1974-422d-45c0-b801-ec1891c3a855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:26.926414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 704/1408] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Sep 14 01:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:27.213472 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:27.215196 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:27.379559 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb813b4-4783-43e6-a8a6-761bec637dd3 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:27.454171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:27.502893 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:27.503499 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:27.506052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:27.506052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:27.506052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:27.609638 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:27.611459 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca21320-dabb-48d9-b618-1d836015d8c7 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:27.630128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 705/1409] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:36.924907 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=79 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:37.627824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=64 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:40.848514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48473-a41b-438e-9376-6cfeba64ceba None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:40.849166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48473-a41b-438e-9376-6cfeba64ceba None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:40.849166 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad48473-a41b-438e-9376-6cfeba64ceba None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:40.850481 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 705/1410] 10.222.0.26 () {54 vars in 918 bytes} [Wed Sep 14 01:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:40.855818 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00042f5d-00a1-4f59-8882-b428dbb73171 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:40.856265 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00042f5d-00a1-4f59-8882-b428dbb73171 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:40.856370 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00042f5d-00a1-4f59-8882-b428dbb73171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:40.937760 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-00042f5d-00a1-4f59-8882-b428dbb73171 None None] MFA Rules not processed for user `ce6a85b7c0c14f76a471680d63078065`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:40.939036 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-00042f5d-00a1-4f59-8882-b428dbb73171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:41.062521 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 706/1411] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 14 01:24:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:41.079332 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d310-3665-4b0b-9684-85fe3a07e551 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:41.079630 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d310-3665-4b0b-9684-85fe3a07e551 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:41.079894 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b0d310-3665-4b0b-9684-85fe3a07e551 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:41.081231 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 706/1412] 10.222.0.26 () {54 vars in 918 bytes} [Wed Sep 14 01:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:41.086964 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0e285-7242-4818-a02f-9c0946082b97 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:41.087296 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0e285-7242-4818-a02f-9c0946082b97 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:41.087643 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a0e285-7242-4818-a02f-9c0946082b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:41.169371 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:41.172660 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-68a0e285-7242-4818-a02f-9c0946082b97 None None] MFA Rules not processed for user `6f6158bf6948476a9244d0c28e1c8404`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:41.174578 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-68a0e285-7242-4818-a02f-9c0946082b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:41.292968 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 707/1413] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Sep 14 01:24:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:43.830896 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:43.831499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:43.831499 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:43.832800 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 707/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:24:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:43.839532 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7753e-e4a1-47a5-8772-296d0dc43bd8 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:43.839990 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7753e-e4a1-47a5-8772-296d0dc43bd8 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:43.840103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f7753e-e4a1-47a5-8772-296d0dc43bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:43.913773 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-67f7753e-e4a1-47a5-8772-296d0dc43bd8 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:43.915028 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-67f7753e-e4a1-47a5-8772-296d0dc43bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:43.975414 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:44.018772 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 708/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:24:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:44.309273 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:44.311130 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:44.475652 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f196e8-0f59-4b8d-b2fc-9c530492c9eb None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:44.551445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:44.593037 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:44.593668 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:44.593933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:44.594235 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:44.595802 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:44.686382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:44.687982 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac060af-16ad-4a3b-a48b-a87abdc06d76 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:44.706409 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 708/1416] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:45.842304 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:45.843210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:45.843210 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:45.845382 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 709/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:45.852635 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eed15a-f471-4d25-9740-7e2bdd40819e None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:45.853283 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eed15a-f471-4d25-9740-7e2bdd40819e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:45.853393 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eed15a-f471-4d25-9740-7e2bdd40819e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:45.940265 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-60eed15a-f471-4d25-9740-7e2bdd40819e None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:45.941518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-60eed15a-f471-4d25-9740-7e2bdd40819e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:45.970486 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:46.042225 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 709/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:46.194409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:46.196903 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:46.357788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087af0af-5c0c-4779-845c-49edf1e1511b None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:46.400774 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:46.477331 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:46.477891 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:46.478188 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:46.478449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:46.480090 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:46.572065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:46.573681 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e4504e-ad31-46eb-88da-b38e1661e0f0 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:46.591866 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 710/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:47.709870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:47.710518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:47.710518 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:47.711809 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 710/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:47.718407 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcec90-b7a8-4469-a1c3-57d6a223f5b9 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:47.718910 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcec90-b7a8-4469-a1c3-57d6a223f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:47.718910 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbcec90-b7a8-4469-a1c3-57d6a223f5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:47.774066 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:47.797205 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-ebbcec90-b7a8-4469-a1c3-57d6a223f5b9 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:47.798495 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-ebbcec90-b7a8-4469-a1c3-57d6a223f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:47.898948 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 711/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:48.052202 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:48.053794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:48.209334 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a49ba5-da61-4a06-9e4c-305d89de27ac None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:48.221861 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:48.335161 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:48.336037 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:48.336445 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:48.336805 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:48.339386 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:48.447553 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:48.449287 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9327e7c5-1830-45bf-8ecb-67555a9c21a3 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:48.468715 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 711/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:49.590278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:49.590981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:49.590981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:49.592189 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 712/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:49.599632 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d1ef-20cf-4100-8872-5601ac5e11c7 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:49.600364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d1ef-20cf-4100-8872-5601ac5e11c7 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:49.600364 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5619d1ef-20cf-4100-8872-5601ac5e11c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:49.628840 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:49.685948 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-5619d1ef-20cf-4100-8872-5601ac5e11c7 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:49.687186 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-5619d1ef-20cf-4100-8872-5601ac5e11c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:49.793383 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 712/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:49.945801 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:49.948233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:50.064021 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:50.102735 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c1ac1c-7078-4fff-9b09-c0b9bbd92738 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:50.225455 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:50.226004 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:50.226293 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:50.226560 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:50.228144 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:50.309828 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:50.330001 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:50.332460 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1c3fa-fda9-456e-aba2-ca5489ca8a34 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:50.353482 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 713/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:51.460243 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:51.461171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:51.461171 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:51.463363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 713/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:51.473788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d374-3fe8-4c45-815c-d1fadeda0f07 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:51.474278 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d374-3fe8-4c45-815c-d1fadeda0f07 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:51.474479 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd4d374-3fe8-4c45-815c-d1fadeda0f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:51.559832 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-afd4d374-3fe8-4c45-815c-d1fadeda0f07 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:51.561041 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-afd4d374-3fe8-4c45-815c-d1fadeda0f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:51.663025 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 714/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:51.820137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:51.821603 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:51.910828 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:51.977213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c2703-5a5b-491a-a03c-0fe480be31a4 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:52.098638 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:52.099254 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:52.099563 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:52.099830 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:52.101465 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:52.151639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:52.194788 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:52.196399 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d9d7b-343d-47bd-8625-9f6426ad1939 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:52.214062 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 714/1428] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:53.270318 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:53.270921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:53.270921 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:53.272287 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 715/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:53.279057 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593f7e-f546-4d93-afcb-b4f29fcc50c4 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:53.279547 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593f7e-f546-4d93-afcb-b4f29fcc50c4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:53.279547 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00593f7e-f546-4d93-afcb-b4f29fcc50c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:53.356675 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-00593f7e-f546-4d93-afcb-b4f29fcc50c4 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:53.357776 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-00593f7e-f546-4d93-afcb-b4f29fcc50c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:53.458621 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 715/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:53.617156 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:53.619574 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:53.674122 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:53.778423 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bcdd63-bb94-4f7d-814e-cd7a78954126 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:53.906107 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:53.906644 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:53.906918 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:53.907186 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:53.908807 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:53.915911 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:54.013871 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:54.015501 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ac369-ae0c-4258-8fb8-2a8b88444cd5 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:54.035811 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 716/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:55.082921 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:55.083736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:55.083736 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:55.085937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 716/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:55.092564 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac1f94-74c3-4e7b-bbfa-e936ea51de27 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:55.093061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac1f94-74c3-4e7b-bbfa-e936ea51de27 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:55.093061 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac1f94-74c3-4e7b-bbfa-e936ea51de27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:55.172399 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-43ac1f94-74c3-4e7b-bbfa-e936ea51de27 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:55.173605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-43ac1f94-74c3-4e7b-bbfa-e936ea51de27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:55.247079 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:55.276128 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 717/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:55.428961 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:55.431424 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:55.455721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:55.590794 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57908c-af90-4306-9c86-1332e41082ad None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:55.687294 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:55.714893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:55.715509 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:55.715781 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:55.716047 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:55.717735 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:55.811841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:55.813462 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf6da5-ddd7-45e2-8233-e1c32752889e service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:55.832013 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 717/1434] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:56.898392 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:56.899320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:56.899320 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:56.901302 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 718/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:56.912318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949830e-f3c4-42f4-8624-82fdc1efefe4 None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:56.912870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949830e-f3c4-42f4-8624-82fdc1efefe4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:56.912870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949830e-f3c4-42f4-8624-82fdc1efefe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:57.004112 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-4949830e-f3c4-42f4-8624-82fdc1efefe4 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:57.005784 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-4949830e-f3c4-42f4-8624-82fdc1efefe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:57.051625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:57.109248 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 718/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:57.261803 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:57.264023 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:57.429747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c18813-7134-4f88-a0a0-a7d29af5d572 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:57.488216 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:57.550557 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:57.551096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:57.551375 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:57.551635 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:57.553231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:57.660977 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:57.662613 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced31369-98b5-4043-a0c8-b2151d990844 service nova] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:57.681725 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 719/1437] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:24:58.722723 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:58.723639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:58.723639 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:58.725870 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 719/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Sep 14 01:24:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:24:58.732347 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d81a1-0581-445c-b366-4749b0782e1a None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:58.732806 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d81a1-0581-445c-b366-4749b0782e1a None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:58.732902 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5d81a1-0581-445c-b366-4749b0782e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:58.811881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-ee5d81a1-0581-445c-b366-4749b0782e1a None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:24:58.813086 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-ee5d81a1-0581-445c-b366-4749b0782e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:58.815496 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:58.912961 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 720/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Sep 14 01:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:24:59.071956 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:24:59.073492 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:59.224404 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3570f4bf-1d48-4ea7-aa78-a26abedb7bd4 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:24:59.251905 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:24:59.347598 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6f6158bf6948476a9244d0c28e1c8404', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:24:59.348122 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:24:59.348387 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:24:59.348674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:24:59.350329 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:24:59.444039 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:24:59.445646 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f90600-cdb4-440b-bb60-43969284cd18 service nova] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:24:59.464633 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 720/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:24:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:01.500976 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:01.501597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:01.501597 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:01.502927 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 721/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:01.509513 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcd271-021e-4419-91d9-7e872f8a3ecb None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:01.510213 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcd271-021e-4419-91d9-7e872f8a3ecb None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:01.510512 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcd271-021e-4419-91d9-7e872f8a3ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:01.556459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:01.583859 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-c8fcd271-021e-4419-91d9-7e872f8a3ecb None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:01.585082 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-c8fcd271-021e-4419-91d9-7e872f8a3ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:01.685875 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 721/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:01.969669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:01.971255 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:02.078065 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692a40ad-76d2-47f4-a7ef-648f5eacaed3 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:02.112498 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:02.156466 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:02.156996 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:02.157271 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:02.157539 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:02.159195 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:02.278383 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:02.279824 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e707ac4-2577-4f34-9913-bd462d0df662 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:02.298761 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 722/1443] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:04.569005 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:04.569629 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:04.569629 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:04.570933 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 722/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:04.577348 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6443-85ab-4f28-be52-745153ed6eec None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:04.578051 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6443-85ab-4f28-be52-745153ed6eec None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:04.578239 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82a6443-85ab-4f28-be52-745153ed6eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:04.651694 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-d82a6443-85ab-4f28-be52-745153ed6eec None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:04.652874 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-d82a6443-85ab-4f28-be52-745153ed6eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:04.666334 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:04.755103 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 723/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:05.043972 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:05.045534 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:05.157507 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc175ad-6475-46d7-9efb-a900ed4034e6 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:05.171253 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:05.238107 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:05.238674 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:05.238944 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:05.239207 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:05.240818 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:05.364441 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:05.365893 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2755ab-027d-4701-a1d9-f8c97d574ef2 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:05.385331 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 723/1446] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:07.822198 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:07.822849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:07.822849 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:07.824817 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 724/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:07.834145 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fafea-e099-48e4-a524-759357ace51b None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:07.834559 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fafea-e099-48e4-a524-759357ace51b None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:07.834734 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4fafea-e099-48e4-a524-759357ace51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:07.886978 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:07.905335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-7a4fafea-e099-48e4-a524-759357ace51b None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:07.906595 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-7a4fafea-e099-48e4-a524-759357ace51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:08.006952 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 724/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:08.288258 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:08.289816 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:08.409468 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8357f4-b5a6-4011-aff9-f94bb2186457 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:08.453208 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:08.488096 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:08.488706 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:08.488982 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:08.489256 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:08.490923 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:08.601522 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:08.603007 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca005f7-0da0-483b-ac60-0884084841af service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:08.621035 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 725/1449] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:10.873418 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:10.874382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:10.874382 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:10.876467 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 725/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:10.885764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d95f-0f28-4a0a-a342-d2e6da576b21 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:10.886223 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d95f-0f28-4a0a-a342-d2e6da576b21 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:10.886359 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e903d95f-0f28-4a0a-a342-d2e6da576b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:10.966998 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-e903d95f-0f28-4a0a-a342-d2e6da576b21 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:10.968252 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-e903d95f-0f28-4a0a-a342-d2e6da576b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:10.991410 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:11.070042 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 726/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:11.356302 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:11.358953 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:11.479224 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72371170-8eba-41a5-8fc2-c17f6215104e None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:11.506522 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:11.561190 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:11.561796 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:11.562095 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:11.562366 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:11.564327 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:11.678657 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:11.680090 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61312565-4f5f-43be-b8d0-98c3a42e7de2 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:11.698770 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 726/1452] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:13.914542 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:13.915159 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:13.915159 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:13.916440 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 727/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:13.921937 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbd8a-d4c7-4f28-8d16-d3c1db3e2a3f None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:13.922421 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbd8a-d4c7-4f28-8d16-d3c1db3e2a3f None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:13.922532 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bdbd8a-d4c7-4f28-8d16-d3c1db3e2a3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:13.987884 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:13.991790 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-15bdbd8a-d4c7-4f28-8d16-d3c1db3e2a3f None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:13.994318 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-15bdbd8a-d4c7-4f28-8d16-d3c1db3e2a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:14.101170 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 727/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:14.388085 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:14.390558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:14.505848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d341b2-5147-4e8b-a2c3-651f3b376d73 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:14.557231 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:14.582931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:14.583547 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:14.583815 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:14.584080 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:14.585669 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:14.697881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:14.699449 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6079b2c-4092-44cb-adb1-69c6eb0019a3 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:14.841394 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 728/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:16.999345 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:17.000218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:17.000218 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:17.002363 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 728/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:17.010965 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f2ac-0954-4f23-9bb3-50b02020ab2c None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:17.011409 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f2ac-0954-4f23-9bb3-50b02020ab2c None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:17.011515 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f2ac-0954-4f23-9bb3-50b02020ab2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:17.081357 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b4f8f2ac-0954-4f23-9bb3-50b02020ab2c None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:17.082605 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b4f8f2ac-0954-4f23-9bb3-50b02020ab2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:17.113233 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:17.182753 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 729/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:25:17.469163 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:17.470761 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:17.576709 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253badc9-a4a1-4702-ad8a-0baf2908c3db None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:17.605080 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:17.650841 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:17.651446 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:17.651714 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:17.651969 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:17.653552 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:17.770995 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:17.772335 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e766a86b-2874-4f0e-9c74-95b22e4cb96d service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:17.790649 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 729/1458] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:19.980561 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:19.981136 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:19.981136 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:19.982439 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 730/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:19.988903 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ea80c-bd3f-4344-9418-88280e16face None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:19.989625 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ea80c-bd3f-4344-9418-88280e16face None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:19.989780 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ea80c-bd3f-4344-9418-88280e16face None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:20.069922 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-981ea80c-bd3f-4344-9418-88280e16face None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:20.071847 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-981ea80c-bd3f-4344-9418-88280e16face None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:20.081670 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:20.170147 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 730/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:20.456881 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:20.459323 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:20.568868 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092ad9bd-6e0f-4deb-b09e-dfb25e70b730 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:20.624720 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:20.644171 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:20.644777 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:20.645047 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:20.645280 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:20.646937 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:20.778738 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:20.780980 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473e17a-afdf-437a-81f2-9a294253cd9d service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:20.804111 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 731/1461] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:23.024227 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:23.024832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:23.024832 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:23.026978 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 731/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:23.035052 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2542-6262-4ff1-a3e2-2a27b8e07f02 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:23.035526 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2542-6262-4ff1-a3e2-2a27b8e07f02 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:23.035633 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64b2542-6262-4ff1-a3e2-2a27b8e07f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:23.106464 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-b64b2542-6262-4ff1-a3e2-2a27b8e07f02 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:23.107585 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-b64b2542-6262-4ff1-a3e2-2a27b8e07f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:23.145788 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:23.208187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 732/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 14 01:25:23.493311 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:23.495014 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:23.611537 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1104c-487c-4c52-a36d-c35fc43f64df None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:23.655075 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:23.692584 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:23.693192 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:23.693460 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:23.693721 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:23.695343 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:23.809137 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:23.810636 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5adb13b-073f-4964-b2c9-106c68d87fb9 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:23.829677 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 732/1464] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:25.988823 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:25.989448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:25.989448 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] PATH_INFO: `/` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:25.990747 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 733/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 14 01:25:25.997301 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b96b82-3fa3-445e-b104-dff426b40bea None None] REQUEST_METHOD: `POST` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:25.998067 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b96b82-3fa3-445e-b104-dff426b40bea None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:25.998265 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b96b82-3fa3-445e-b104-dff426b40bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:26.071706 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.auth.core [None req-f3b96b82-3fa3-445e-b104-dff426b40bea None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81415) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:26.073073 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-f3b96b82-3fa3-445e-b104-dff426b40bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:26.093245 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:26.180982 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 733/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:26.468276 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] Authenticating user token {{(pid=81414) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:26.469820 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:26.578981 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fa406e-aefc-4d26-9367-3b7fd9e1ff76 None None] Validating token access rules against request {{(pid=81414) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:26.647361 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:26.656008 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81414) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:26.656616 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] REQUEST_METHOD: `GET` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:26.656886 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:26.657145 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:26.658764 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:26.772411 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:26.773763 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c35b09-2c4e-4c7c-afa7-ffeda8555af1 service neutron] RBAC: Authorization granted {{(pid=81414) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:26.792888 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 734/1467] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:28.975453 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:28.976027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:28.976027 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] PATH_INFO: `/` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:28.977357 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 734/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Sep 14 01:25:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 14 01:25:28.984154 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4ffc4-f404-4fc6-9579-c008304efd63 None None] REQUEST_METHOD: `POST` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:28.984931 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4ffc4-f404-4fc6-9579-c008304efd63 None None] SCRIPT_NAME: `/identity` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:28.985187 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e4ffc4-f404-4fc6-9579-c008304efd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81414) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:29.062558 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.auth.core [None req-f5e4ffc4-f404-4fc6-9579-c008304efd63 None None] MFA Rules not processed for user `f48f44237b4c4aa48da7b17eda7c3281`. Rule list: `[]` (Enabled: `True`). {{(pid=81414) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 14 01:25:29.064127 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG keystone.common.fernet_utils [None req-f5e4ffc4-f404-4fc6-9579-c008304efd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81414) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:29.113048 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=100 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:29.161425 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: [pid: 81414|app: 0|req: 735/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Sep 14 01:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 14 01:25:29.445237 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] Authenticating user token {{(pid=81415) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 14 01:25:29.447835 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:29.557414 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0467095f-c780-42bf-ac3c-8d0afe582a71 None None] Validating token access rules against request {{(pid=81415) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 14 01:25:29.608904 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=100 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:29.636459 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1f5db3c037944dfbb2e4dd1bd8fe24b3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9f08639310641c5a5070068ee6f1c76', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81415) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 14 01:25:29.637061 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] REQUEST_METHOD: `GET` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 14 01:25:29.637333 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] SCRIPT_NAME: `/identity` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 14 01:25:29.637597 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81415) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 14 01:25:29.639211 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.fernet_utils [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81415) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 14 01:25:29.763152 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 14 01:25:29.764514 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1ef625-09b4-42c4-8a18-ba28ee0c4305 service neutron] RBAC: Authorization granted {{(pid=81415) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 14 01:25:29.785153 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: [pid: 81415|app: 0|req: 735/1470] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Sep 14 01:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 14 01:25:39.159848 dv-cinder-857075-4-iscsi devstack@keystone.service[81414]: DEBUG dbcounter [-] [81414] Writing DB stats keystone:SELECT=22 {{(pid=81414) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 14 01:25:39.782576 dv-cinder-857075-4-iscsi devstack@keystone.service[81415]: DEBUG dbcounter [-] [81415] Writing DB stats keystone:SELECT=66 {{(pid=81415) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}